-- Logs begin at Mon 2022-09-19 14:11:26 UTC, end at Mon 2022-09-19 15:09:12 UTC. -- Sep 19 14:37:43.835530 dv-cinder-756709-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 19 14:37:43.843692 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 19 14:37:43.846277 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 19 14:37:43 2022] *** Sep 19 14:37:43.846417 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 19 14:37:43.846538 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 19 14:37:43.846638 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: nodename: dv-cinder-756709-7-iscsi Sep 19 14:37:43.846739 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: machine: x86_64 Sep 19 14:37:43.846844 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: clock source: unix Sep 19 14:37:43.846944 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: pcre jit disabled Sep 19 14:37:43.847046 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: detected number of CPU cores: 4 Sep 19 14:37:43.847149 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: current working directory: / Sep 19 14:37:43.847249 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: detected binary path: /usr/bin/uwsgi-core Sep 19 14:37:43.847350 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: your processes number limit is 47680 Sep 19 14:37:43.847453 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: your memory page size is 4096 bytes Sep 19 14:37:43.847556 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: detected max file descriptor number: 2048 Sep 19 14:37:43.847664 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: lock engine: pthread robust mutexes Sep 19 14:37:43.847936 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: thunder lock: enabled Sep 19 14:37:43.848244 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 19 14:37:43.848423 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 19 14:37:43.887684 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: Python main interpreter initialized at 0x5602363599c0 Sep 19 14:37:43.887684 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: python threads support enabled Sep 19 14:37:43.887684 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: your server socket listen backlog is limited to 100 connections Sep 19 14:37:43.887684 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: your mercy for graceful operations on workers is 90 seconds Sep 19 14:37:43.888123 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: mapped 403077 bytes (393 KB) for 2 cores Sep 19 14:37:43.888123 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: *** Operational MODE: preforking *** Sep 19 14:37:43.888404 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: *** uWSGI is running in multiple interpreter mode *** Sep 19 14:37:43.888404 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: spawned uWSGI master process (pid: 87860) Sep 19 14:37:43.888622 dv-cinder-756709-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 19 14:37:43.889860 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: spawned uWSGI worker 1 (pid: 87872, cores: 1) Sep 19 14:37:43.889860 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: spawned uWSGI worker 2 (pid: 87873, cores: 1) Sep 19 14:37:43.889860 dv-cinder-756709-7-iscsi devstack@keystone.service[87860]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 19 14:37:48.167604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 14:37:48.167604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: from cryptography.utils import int_from_bytes Sep 19 14:37:48.168449 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 14:37:48.168449 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: from cryptography.utils import int_from_bytes Sep 19 14:37:48.169273 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 14:37:48.169273 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: from cryptography.utils import int_from_bytes Sep 19 14:37:48.169736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 14:37:48.169736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: from cryptography.utils import int_from_bytes Sep 19 14:37:48.218862 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 14:37:48.219064 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 14:37:48.219155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 14:37:48.219242 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 14:37:48.219316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] command line args: [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 14:37:48.219387 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] command line args: [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 14:37:48.219468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 14:37:48.219540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 14:37:48.219655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ================================================================================ {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 14:37:48.219740 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ================================================================================ {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 14:37:48.219939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] admin_token = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.220045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] admin_token = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.220190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] config_dir = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.220276 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] config_dir = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.220354 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.220424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.220539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] config_source = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.220627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] config_source = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.220706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] debug = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.220778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] debug = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.220901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.222265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.222265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.222503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.222578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.222694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.222694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.222879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.222960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.223031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.223152 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] list_limit = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.223228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] list_limit = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.223342 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] log_config_append = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.223425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] log_config_append = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.223497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.223611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.223685 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] log_dir = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.223805 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] log_dir = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.223916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] log_file = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.224003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] log_file = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.224082 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.224204 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.224285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.224362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.224483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.224568 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.224640 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.224763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.224840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.224962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.225046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.225124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.225255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.225332 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.225446 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.225530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.225609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.225724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.225804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.225881 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.225995 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.226076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.226158 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.226278 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.226359 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.226436 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.226550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.226625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.226745 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.226822 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.226943 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.227019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.227144 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] publish_errors = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.227219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] publish_errors = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.227295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.227372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.227454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.227531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.227602 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.227676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.227747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.227855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.227987 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.228069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.228140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] standard_threads = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.228216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] standard_threads = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.228295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.228373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.228450 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.228558 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.228637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.228715 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.228836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] use_journal = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.228917 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] use_json = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.228999 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] use_journal = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.229561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] use_stderr = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.229704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] use_json = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.229840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] use_syslog = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.229968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] use_stderr = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.229968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] use_syslog = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.230104 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.230180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 14:37:48.230257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.230398 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.230485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.230641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.230719 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.230875 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.230953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.231121 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.231207 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.231356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.231436 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.231601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.231680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.231847 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.231943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.232105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.232184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.232347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.232424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.232594 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.232684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.232824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.232907 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.233067 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.233145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.233301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.233387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.233537 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.233616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.233779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.233857 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.234012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.234099 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.234252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.234341 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.234509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.234586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.234753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.234909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.234990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.235062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.235224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.235310 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.235480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.235559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.235730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.235808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.236010 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.236129 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.236254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.236369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.236481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.236600 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.236713 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.236835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.236962 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.237076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.237195 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.237308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.237419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.237538 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.237659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.237772 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.237890 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.238010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.238123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.238235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.238354 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.238473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.238587 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.238698 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.238810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.238922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.239042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.239155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.239274 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.239398 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.239512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.239630 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.239751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.239913 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.240029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.240142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.240264 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.240379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.240567 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.240684 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.240796 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.240909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.241020 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.241133 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.241254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.241374 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.241487 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.241599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.241710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.241822 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.241945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.242049 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.242145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.242245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.242382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.242477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.242582 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.242687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.242781 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.242875 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.242969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.243069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.243182 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.243294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.243406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.243541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.243663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.243782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.243910 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.244035 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.244150 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.244264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.244376 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.244500 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.244615 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.244724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.245455 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.245585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.245701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.245822 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.245951 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.245951 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.246174 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.246286 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.246400 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.246512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.246625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.246746 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.246860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.246973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.247085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.247198 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.247313 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.247440 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.247553 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.247666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.247785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.247946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.248076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.248200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.248314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.248428 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.248541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.248653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.248774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.248896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.249010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.249123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.249242 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.249356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.249476 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.249602 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.249722 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.249836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.249955 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.250068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.250181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.250304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.250419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.250533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.250637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.250748 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.250860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.250981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.251095 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.251207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.251325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.251439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.251553 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.251674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.251788 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.251911 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.252023 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.252136 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.252258 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.252485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.252606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.252720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.252833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.252946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.253068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.253190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.253317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.253438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.253559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.253675 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.253789 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.253886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.254009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.254131 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.254251 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.254366 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.254480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.254594 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.254716 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.254831 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.254944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.255057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.255170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.255284 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.255405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.255519 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.255633 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.255753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.255917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.256046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] auth.external = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.256191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.256314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.256437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.256553 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.256667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] auth.external = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.256781 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.256904 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.257018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] auth.password = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.257018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] auth.token = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.257242 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.257360 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.257475 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.257589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.257704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] auth.password = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.257825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.257942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] auth.token = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.258055 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.258169 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.258282 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.258396 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.258518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.258633 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.258747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.258860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.258974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] credential.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.259088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.259210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.259325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.259445 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.259560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.259674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.259799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] credential.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.259941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.260058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.260172 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.260293 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.260408 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.260522 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.260643 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.260759 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.260873 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.260987 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.261101 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.261215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.261337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.261452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.261566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.261680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.261794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.261907 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.262029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.262145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.262265 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.262380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.262524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.262642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.262765 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.262880 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.262993 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.263108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.263221 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.263335 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.263457 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.263572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.263686 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.263800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.263933 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.264060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.264177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.264300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.264414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.264528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.264648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.264772 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.264894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.265009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.265123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.265244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.265359 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.265481 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.265596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.265709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.265813 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.265908 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.266003 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.266187 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.266311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.266462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.266580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.266694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.266808 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.267033 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.267157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.267273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.267392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.267506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.267621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.267621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.267902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.268021 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.268136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.268251 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.268367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.268367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.268591 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.268591 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.268805 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.268919 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.269033 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.269155 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.269271 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.269386 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.269507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.269622 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.269736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.269859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.269974 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.270088 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.270202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.270315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.270430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.270553 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.270668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.270782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.270896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.271009 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.271122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.271245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.271360 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.271475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.271588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.271701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.271815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.272011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.272132 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.272246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.272367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.272482 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.272690 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.272818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.272935 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.273050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.273165 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.273279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.273393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.273516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.273633 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.273747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.273861 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.273975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.274097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.274213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.274327 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.274441 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.274441 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.274656 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.274778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.274894 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.275008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.275122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.275236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.275350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.275472 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.275588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.275588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.275803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.275964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.276079 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.276206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.276323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.276437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.276552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.276666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.276781 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.276781 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.277005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.277120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.277234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.277351 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.277465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.277587 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.277702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.277817 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.277931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.278046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.278166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.278290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.278405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.278518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.278632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.278746 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.278860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.278982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.279098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.279212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.279325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.279439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.279553 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.279674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.279790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.279922 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.280036 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.280150 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.280264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.280389 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.280505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.280619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.280734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.280847 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.280962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.281084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.281200 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.281313 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.281427 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.281540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.281654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.281776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.281891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.282006 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.282120 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.282233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.282347 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.282469 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.282584 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.282738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.282854 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.282967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.283080 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.283201 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.283315 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.283428 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.283540 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.283653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.283765 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.283949 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.284067 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.284180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.284292 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.284405 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.284517 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.284638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.284638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.284851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.285077 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.285190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.285312 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.285426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.285538 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.285651 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.285763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.285876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.286002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.286118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.286231 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.286344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.286457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.286570 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.286691 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.286812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.286926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.287039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.287151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.287268 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.287391 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.287505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.287617 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.287730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.287861 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.287977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.288093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.288250 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.288367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.288482 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] role.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.288596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.288784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] role.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.288902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.289016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.289130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.289244 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.289359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.289481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.289598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.289711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.289825 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.289939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.290053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] role.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.290175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.290290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] role.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.290405 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.290503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.290599 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.290712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.290833 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.290949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.291063 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.291177 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.291291 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.291406 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.291527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.292421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.292538 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.292652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.292798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.292915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.292915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.293145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.293259 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.293373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.293487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.293601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.293724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.293839 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.293839 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.294053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.294167 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.294281 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.294404 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.294520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.294634 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.294748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.294862 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.294984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.295099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.295213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.295327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.295441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.295555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.295555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.295779 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.295905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.296020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.296133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.296247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.296372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.296488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.296602 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.296718 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.296839 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.296955 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.297077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.297192 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.297309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] token.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.297424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.297538 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.297652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.297776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.297891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.298005 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.298119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.298232 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.298346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.298468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.298583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.298697 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.298812 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.298926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.299040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] token.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.299161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.299277 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.299390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.299489 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.299584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.299697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.299864 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.299984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.300098 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.300212 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.300326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.300440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.300566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.300681 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.300795 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.300919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.301042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.301158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.301271 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.301393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.301508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.connection = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.301622 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.301736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.301850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.301964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.302087 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.302202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.302316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.302429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.302544 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.302666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.302826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.302949 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.303073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.303189 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.303303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.connection = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.303417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.303531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.303644 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.303767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.303897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.304013 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.304127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.304318 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.304437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.304563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.304679 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.304793 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.304906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.305019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.305132 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.305254 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.305370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.305370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.305582 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.305695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.305808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.305930 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 14:37:48.306045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.306045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.306045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.306045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.306045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.306045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.306045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.306045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 14:37:48.306856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 14:37:48.333599 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.333824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.334047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.334731 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.334966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.335193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.342444 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.343576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.371023 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.371249 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.372507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.372772 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.423776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.424512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.471805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.472055 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.473362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.473600 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.628142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.628309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.628432 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.628521 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.628671 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.628759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.628899 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.628983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.629107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.629202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.629329 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.629547 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.652294 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.652516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.652726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.652839 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.652949 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.653036 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.653166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.653252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.653379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.653467 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.653589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.653675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.653804 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.654024 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.665034 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.665142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.678292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.678522 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.678677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.678792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.678917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.679004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.679150 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.679235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.679606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.679687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.679764 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.679862 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.679862 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.680015 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 19 14:37:48.696482 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.696689 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 14:37:48.697047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.697141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.697219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 14:37:48.697358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 14:37:48.697493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.697702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.697787 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.698027 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 19 14:37:48.698127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.698303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.698387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.698572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.698836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.699099 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.707791 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.708089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.708359 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.708641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.708725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.708972 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.709249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.709536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.716046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.717127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.718904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.719191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.719470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.719746 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.720041 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.720125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.720297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.720381 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.720564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.720647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.720835 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.720925 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.721104 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.721186 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.721376 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.721458 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.721642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.721725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.721917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.722193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.722474 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.722744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.744218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.746247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.746352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.746533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.747954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.748103 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.751367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.751649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.751964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.752751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.753038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.753321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.758294 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.758579 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.759749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.760068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.764892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.765189 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.765477 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.765769 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.766043 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.766316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.766559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.766657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.766796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.766900 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.767094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.767390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.767677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.767988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.768265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.768538 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.782082 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.782357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.783969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.784253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.786150 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.788076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.789333 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.789626 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.791262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.791564 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.794004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.796023 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.797191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.799253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.800388 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.800690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.800974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.801253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.801537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.801826 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.802500 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.802805 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.803095 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.803382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.803673 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.803979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.817911 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.818196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.818461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.818722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.820251 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.820543 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.820815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.821305 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.826296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.827952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.829215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.829523 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.829797 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.830768 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.832345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.832468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.832626 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.832754 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.833041 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.833322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.833719 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.835356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.835645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.835952 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.836243 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.836652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.844721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.845010 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.845289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.845567 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.845944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.846236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.846511 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.847796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.848105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.848387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.848691 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.849055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.849345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.849638 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.862164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.865065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.865494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.868096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.868427 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.869588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.871112 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.871386 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.871486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.871784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.872999 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.874472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.874759 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.875173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.878137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.879641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.881572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.882464 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.882857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.883029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.883190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.883395 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.886498 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.886896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.887187 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.887470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.891646 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.891963 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.892244 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.895281 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.895579 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.895903 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.899040 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.899321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.899604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.899896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.900180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.900451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.900725 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.901657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.902035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.902340 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.902619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.902887 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.903151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.903392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=87873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 19 14:37:48.903485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.903682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.903999 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.904283 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.904559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.904842 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.905134 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.905417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.905700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.905985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.906264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.906527 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.906893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.907240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 19 14:37:48.937325 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5602363599c0 pid: 87872 (default app) Sep 19 14:37:48.940659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227db41-bea2-4f30-9d39-ba86d4fa99a1 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:37:48.940946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227db41-bea2-4f30-9d39-ba86d4fa99a1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:37:48.941220 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227db41-bea2-4f30-9d39-ba86d4fa99a1 None None] PATH_INFO: `/v3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:37:48.941422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5602363599c0 pid: 87873 (default app) Sep 19 14:37:48.945350 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1/1] 10.222.0.60 () {50 vars in 791 bytes} [Mon Sep 19 14:37:48 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:37:55.899216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396371d-6b58-48f2-8bc1-49c988333191 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:37:55.899758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396371d-6b58-48f2-8bc1-49c988333191 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:37:55.899758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396371d-6b58-48f2-8bc1-49c988333191 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:37:55.909030 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1/2] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:37:55 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:37:55.915054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365d7ab-258b-47cc-863e-48ac470c4dcb None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:37:55.915636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365d7ab-258b-47cc-863e-48ac470c4dcb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:37:55.915636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365d7ab-258b-47cc-863e-48ac470c4dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:37:55.991653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: INFO dbcounter [None req-2365d7ab-258b-47cc-863e-48ac470c4dcb None None] Registered counter for database keystone Sep 19 14:37:56.011232 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG oslo_db.sqlalchemy.engines [None req-2365d7ab-258b-47cc-863e-48ac470c4dcb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87872) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 14:37:56.017510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writer thread running {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 14:37:56.196462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG passlib.handlers.bcrypt [None req-2365d7ab-258b-47cc-863e-48ac470c4dcb None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87872) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 19 14:37:56.196776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG passlib.handlers.bcrypt [None req-2365d7ab-258b-47cc-863e-48ac470c4dcb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87872) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 19 14:37:56.257318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2365d7ab-258b-47cc-863e-48ac470c4dcb None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:37:56.258969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2365d7ab-258b-47cc-863e-48ac470c4dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:37:56.391645 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 2/3] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:37:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 19 14:37:56.847167 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396371d-6b58-48f2-8bc1-49c988333191 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:37:56.917534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: INFO dbcounter [None req-5396371d-6b58-48f2-8bc1-49c988333191 None None] Registered counter for database keystone Sep 19 14:37:56.939733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG oslo_db.sqlalchemy.engines [None req-5396371d-6b58-48f2-8bc1-49c988333191 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87873) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 14:37:56.945934 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writer thread running {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 14:37:57.066558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396371d-6b58-48f2-8bc1-49c988333191 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:37:57.070908 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74678b43-7c4a-4a7c-93cb-e46433b725c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:37:57.071536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74678b43-7c4a-4a7c-93cb-e46433b725c2 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:37:57.071845 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74678b43-7c4a-4a7c-93cb-e46433b725c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:37:57.072125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74678b43-7c4a-4a7c-93cb-e46433b725c2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:37:57.080721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74678b43-7c4a-4a7c-93cb-e46433b725c2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:37:57.132842 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: WARNING keystone.common.rbac_enforcer.enforcer [None req-74678b43-7c4a-4a7c-93cb-e46433b725c2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 19 14:37:57.133160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74678b43-7c4a-4a7c-93cb-e46433b725c2 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:37:57.143231 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 2/4] 10.222.0.60 () {58 vars in 1324 bytes} [Mon Sep 19 14:37:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 14:37:57.149186 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365d7ab-258b-47cc-863e-48ac470c4dcb None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:37:57.166271 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365d7ab-258b-47cc-863e-48ac470c4dcb None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:37:57.173285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f748a02-ef3d-4697-acb3-0a48ca16a727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:37:57.175160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f748a02-ef3d-4697-acb3-0a48ca16a727 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:37:57.175688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f748a02-ef3d-4697-acb3-0a48ca16a727 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:37:57.176121 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f748a02-ef3d-4697-acb3-0a48ca16a727 admin admin] PATH_INFO: `/v3/projects/b206fdda52da4f339c6c008d4c7973af` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:37:57.177761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f748a02-ef3d-4697-acb3-0a48ca16a727 admin admin] RBAC: Authorizing `identity:get_project(project_id=b206fdda52da4f339c6c008d4c7973af)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:37:57.230692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2f748a02-ef3d-4697-acb3-0a48ca16a727 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 19 14:37:57.231032 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f748a02-ef3d-4697-acb3-0a48ca16a727 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:37:57.235591 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 3/5] 10.222.0.60 () {56 vars in 1207 bytes} [Mon Sep 19 14:37:57 2022] GET /identity/v3/projects/b206fdda52da4f339c6c008d4c7973af => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:37:59.357137 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e43ec-fdef-49aa-88bc-0439ec5a463a None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:37:59.357649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e43ec-fdef-49aa-88bc-0439ec5a463a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:37:59.357649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e43ec-fdef-49aa-88bc-0439ec5a463a None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:37:59.361701 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 3/6] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:37:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:37:59.367138 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944d473-0257-4899-9fd6-efcb9f0be224 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:37:59.367585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944d473-0257-4899-9fd6-efcb9f0be224 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:37:59.367704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944d473-0257-4899-9fd6-efcb9f0be224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:37:59.388425 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1944d473-0257-4899-9fd6-efcb9f0be224 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:37:59.389920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1944d473-0257-4899-9fd6-efcb9f0be224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:37:59.397936 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 4/7] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:37:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 19 14:37:59.661382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e43ec-fdef-49aa-88bc-0439ec5a463a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:37:59.673664 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e43ec-fdef-49aa-88bc-0439ec5a463a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:37:59.677876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fa694-0602-41ce-b106-40b186de6984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:37:59.678474 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fa694-0602-41ce-b106-40b186de6984 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:37:59.678753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fa694-0602-41ce-b106-40b186de6984 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:37:59.679025 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fa694-0602-41ce-b106-40b186de6984 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:37:59.687362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fa694-0602-41ce-b106-40b186de6984 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:37:59.689172 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fa694-0602-41ce-b106-40b186de6984 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:37:59.697734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 4/8] 10.222.0.60 () {58 vars in 1324 bytes} [Mon Sep 19 14:37:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 14:37:59.703805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944d473-0257-4899-9fd6-efcb9f0be224 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:37:59.712854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944d473-0257-4899-9fd6-efcb9f0be224 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:37:59.717254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc39d0f-c0f0-4dd0-a837-ad3b4ad47e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:37:59.717880 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc39d0f-c0f0-4dd0-a837-ad3b4ad47e39 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:37:59.718167 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc39d0f-c0f0-4dd0-a837-ad3b4ad47e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:37:59.718443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc39d0f-c0f0-4dd0-a837-ad3b4ad47e39 admin admin] PATH_INFO: `/v3/users/7d770189a7bb4b81b8a42702f35fc84d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:37:59.719752 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc39d0f-c0f0-4dd0-a837-ad3b4ad47e39 admin admin] RBAC: Authorizing `identity:get_user(user_id=7d770189a7bb4b81b8a42702f35fc84d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:37:59.721619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc39d0f-c0f0-4dd0-a837-ad3b4ad47e39 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:37:59.723178 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 5/9] 10.222.0.60 () {56 vars in 1198 bytes} [Mon Sep 19 14:37:59 2022] GET /identity/v3/users/7d770189a7bb4b81b8a42702f35fc84d => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:01.955062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0cbdf-5c71-424b-9068-756c40235f3a None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:01.955586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0cbdf-5c71-424b-9068-756c40235f3a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:01.955586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc0cbdf-5c71-424b-9068-756c40235f3a None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:01.957121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 5/10] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:01.962286 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6b252-859a-431a-80a2-a45ea287b551 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:01.962691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6b252-859a-431a-80a2-a45ea287b551 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:01.962843 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6b252-859a-431a-80a2-a45ea287b551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:01.981552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e2c6b252-859a-431a-80a2-a45ea287b551 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:01.982749 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e2c6b252-859a-431a-80a2-a45ea287b551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:01.991744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7913d3-7055-4321-982d-68b306c4b72d None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:01.992076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7913d3-7055-4321-982d-68b306c4b72d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:01.992352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7913d3-7055-4321-982d-68b306c4b72d None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:01.993667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 6/11] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:38:01.999316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f99b6-d817-4c3c-ac5f-ea48dcb58adb None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:01.999602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f99b6-d817-4c3c-ac5f-ea48dcb58adb None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:01.999908 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f99b6-d817-4c3c-ac5f-ea48dcb58adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:02.040987 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG passlib.handlers.bcrypt [None req-8c3f99b6-d817-4c3c-ac5f-ea48dcb58adb None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=87873) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 19 14:38:02.041389 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG passlib.handlers.bcrypt [None req-8c3f99b6-d817-4c3c-ac5f-ea48dcb58adb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87873) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 19 14:38:02.066948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 6/12] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:02.091129 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8c3f99b6-d817-4c3c-ac5f-ea48dcb58adb None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:02.092832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8c3f99b6-d817-4c3c-ac5f-ea48dcb58adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:02.198405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 7/13] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:02.334479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6b252-859a-431a-80a2-a45ea287b551 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:02.358496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:02.440332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6b252-859a-431a-80a2-a45ea287b551 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:02.466463 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f99b6-d817-4c3c-ac5f-ea48dcb58adb None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:02.474041 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429bc4a-3a9f-4738-9f83-a5c388be3a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:02.474698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429bc4a-3a9f-4738-9f83-a5c388be3a51 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:02.474983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429bc4a-3a9f-4738-9f83-a5c388be3a51 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:02.475262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429bc4a-3a9f-4738-9f83-a5c388be3a51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:02.486573 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429bc4a-3a9f-4738-9f83-a5c388be3a51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-9429bc4a-3a9f-4738-9f83-a5c388be3a51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:02.495898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:02.496845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:02.498303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:02.498303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:02.498303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:02.498303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:02.498303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:02.498303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:02.498303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:02.498303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:02.498303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:02.498303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:02.498303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:02.498303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:02.498303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 7/14] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:38:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:02.502808 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429bc4a-3a9f-4738-9f83-a5c388be3a51 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:02.566539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f99b6-d817-4c3c-ac5f-ea48dcb58adb None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:02.568802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:02.586369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:02.605910 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429bc4a-3a9f-4738-9f83-a5c388be3a51 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:02.606082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a9432-a650-4aa8-a8d7-46b55885d5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:02.606852 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a9432-a650-4aa8-a8d7-46b55885d5d8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:02.607136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a9432-a650-4aa8-a8d7-46b55885d5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:02.607410 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790a9432-a650-4aa8-a8d7-46b55885d5d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:02.641502 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0578cc-2b67-4c78-965a-568edf7ecbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:02.642115 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0578cc-2b67-4c78-965a-568edf7ecbbf None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:02.642401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0578cc-2b67-4c78-965a-568edf7ecbbf None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:02.642679 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0578cc-2b67-4c78-965a-568edf7ecbbf None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:02.643356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0578cc-2b67-4c78-965a-568edf7ecbbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:02.643672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0578cc-2b67-4c78-965a-568edf7ecbbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:02.645122 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0578cc-2b67-4c78-965a-568edf7ecbbf None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:02.656882 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 8/15] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:02.662573 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0578cc-2b67-4c78-965a-568edf7ecbbf None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:02.680274 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a9432-a650-4aa8-a8d7-46b55885d5d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:02.682365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790a9432-a650-4aa8-a8d7-46b55885d5d8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:02.761570 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0578cc-2b67-4c78-965a-568edf7ecbbf None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:02.781077 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 8/16] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 14:38:02.786627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a9432-a650-4aa8-a8d7-46b55885d5d8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:02.789393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:02.798335 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60faeb11-164b-44e3-ae0f-ee94f9c291d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:02.798976 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60faeb11-164b-44e3-ae0f-ee94f9c291d7 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:02.799263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60faeb11-164b-44e3-ae0f-ee94f9c291d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:02.799547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60faeb11-164b-44e3-ae0f-ee94f9c291d7 None admin] PATH_INFO: `/v3/users/7d770189a7bb4b81b8a42702f35fc84d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:02.800740 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60faeb11-164b-44e3-ae0f-ee94f9c291d7 None admin] RBAC: Authorizing `identity:get_user(user_id=7d770189a7bb4b81b8a42702f35fc84d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:02.802217 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60faeb11-164b-44e3-ae0f-ee94f9c291d7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:02.803896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 9/17] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:02 2022] GET /identity/v3/users/7d770189a7bb4b81b8a42702f35fc84d => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:02.809760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60faeb11-164b-44e3-ae0f-ee94f9c291d7 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:02.824010 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:02.890001 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790a9432-a650-4aa8-a8d7-46b55885d5d8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:02.910449 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60faeb11-164b-44e3-ae0f-ee94f9c291d7 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:02.924141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b0952-07d0-4cfd-a027-fe4d4d85b050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:02.925002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b0952-07d0-4cfd-a027-fe4d4d85b050 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:02.925296 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b0952-07d0-4cfd-a027-fe4d4d85b050 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:02.925571 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b0952-07d0-4cfd-a027-fe4d4d85b050 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:02.944973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b0952-07d0-4cfd-a027-fe4d4d85b050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:02.946136 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f6a64-e576-4bcb-b50c-2f0bea9e9fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:02.946428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b0952-07d0-4cfd-a027-fe4d4d85b050 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:02.946764 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f6a64-e576-4bcb-b50c-2f0bea9e9fee None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:02.947049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f6a64-e576-4bcb-b50c-2f0bea9e9fee None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:02.947327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f6a64-e576-4bcb-b50c-2f0bea9e9fee None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:02.948454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f6a64-e576-4bcb-b50c-2f0bea9e9fee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:02.950504 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f6a64-e576-4bcb-b50c-2f0bea9e9fee None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:02.952074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 10/18] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:02.957838 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f6a64-e576-4bcb-b50c-2f0bea9e9fee None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-af4b0952-07d0-4cfd-a027-fe4d4d85b050 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:38:02.965936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:02.967229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:38:02.968234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:38:02.968234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:38:02.968234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:02.968234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 9/19] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:38:02.972928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b0952-07d0-4cfd-a027-fe4d4d85b050 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:03.030309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:03.039637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:03.059153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f6a64-e576-4bcb-b50c-2f0bea9e9fee None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:03.080248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b0952-07d0-4cfd-a027-fe4d4d85b050 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:03.095005 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7b7bab-d333-4854-a3bf-ddfa42bdef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:03.095616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b7bab-d333-4854-a3bf-ddfa42bdef11 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:03.095930 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b7bab-d333-4854-a3bf-ddfa42bdef11 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:03.096203 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7b7bab-d333-4854-a3bf-ddfa42bdef11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:03.096924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b7bab-d333-4854-a3bf-ddfa42bdef11 None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.domain.id=default, user.id=7d770189a7bb4b81b8a42702f35fc84d) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:03.097253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b7bab-d333-4854-a3bf-ddfa42bdef11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:03.098689 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7b7bab-d333-4854-a3bf-ddfa42bdef11 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:03.105283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 11/20] 10.222.0.60 () {56 vars in 1322 bytes} [Mon Sep 19 14:38:02 2022] GET /identity/v3/role_assignments?user.id=7d770189a7bb4b81b8a42702f35fc84d&scope.domain.id=default&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:03.116920 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6539311-857a-41fb-a15d-2370ca1ccd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:03.117558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6539311-857a-41fb-a15d-2370ca1ccd59 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:03.117844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6539311-857a-41fb-a15d-2370ca1ccd59 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:03.118123 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6539311-857a-41fb-a15d-2370ca1ccd59 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:03.118785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6539311-857a-41fb-a15d-2370ca1ccd59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:03.119091 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6539311-857a-41fb-a15d-2370ca1ccd59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:03.121145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6539311-857a-41fb-a15d-2370ca1ccd59 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:03.141599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 10/21] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:05.512143 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52850c44-e16e-4162-8c72-bc882e5abd2e None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:05.513186 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52850c44-e16e-4162-8c72-bc882e5abd2e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:05.513674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52850c44-e16e-4162-8c72-bc882e5abd2e None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:05.515213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 12/22] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:05.521058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf989d5-bb88-4160-8c19-179049439927 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:05.522009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf989d5-bb88-4160-8c19-179049439927 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:05.522492 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf989d5-bb88-4160-8c19-179049439927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:05.552855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-aaf989d5-bb88-4160-8c19-179049439927 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:05.554417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-aaf989d5-bb88-4160-8c19-179049439927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:05.625133 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 11/23] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:06.059127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b85e4a-98c3-4d98-a531-db7a8f6b7ce1 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:06.059809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b85e4a-98c3-4d98-a531-db7a8f6b7ce1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:06.060335 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b85e4a-98c3-4d98-a531-db7a8f6b7ce1 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:06.063142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 13/24] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:06.065208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec94828-1dc4-46a4-b071-0010399dab5d None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:06.065717 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec94828-1dc4-46a4-b071-0010399dab5d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:06.066185 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec94828-1dc4-46a4-b071-0010399dab5d None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:06.070051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 12/25] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:06.075038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e97de-b176-4444-958b-235ce3c9c462 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:06.075614 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e97de-b176-4444-958b-235ce3c9c462 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:06.076190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872e97de-b176-4444-958b-235ce3c9c462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:06.080879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e3a56-c2fe-4837-b3d0-0e9ee444b37c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:06.083338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e3a56-c2fe-4837-b3d0-0e9ee444b37c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:06.085307 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e3a56-c2fe-4837-b3d0-0e9ee444b37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:06.112967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-872e97de-b176-4444-958b-235ce3c9c462 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:06.114210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-872e97de-b176-4444-958b-235ce3c9c462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:06.117062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-de2e3a56-c2fe-4837-b3d0-0e9ee444b37c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:06.118518 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-de2e3a56-c2fe-4837-b3d0-0e9ee444b37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:06.164689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:06.191684 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 13/26] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:06.193079 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e3a56-c2fe-4837-b3d0-0e9ee444b37c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:06.241200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 14/27] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:06.298841 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e3a56-c2fe-4837-b3d0-0e9ee444b37c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:06.334677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7d491-a0cc-42b3-8ead-d08763235c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:06.335640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7d491-a0cc-42b3-8ead-d08763235c23 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:06.336151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7d491-a0cc-42b3-8ead-d08763235c23 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:06.336616 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7d491-a0cc-42b3-8ead-d08763235c23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:06.348729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df7d491-a0cc-42b3-8ead-d08763235c23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-1df7d491-a0cc-42b3-8ead-d08763235c23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:06.355443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:06.356337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:06.357107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:06.357107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:06.357107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:06.357107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:06.357107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:06.357107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:06.357107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:06.357107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:06.357107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:06.357107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:06.357107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:06.357107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:06.359329 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 14/28] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:38:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:06.368957 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e97de-b176-4444-958b-235ce3c9c462 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:06.489839 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:06.526726 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872e97de-b176-4444-958b-235ce3c9c462 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:06.572620 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6846f-d20c-460b-ba8e-2ba9478e80f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:06.573241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6846f-d20c-460b-ba8e-2ba9478e80f7 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:06.573521 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6846f-d20c-460b-ba8e-2ba9478e80f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:06.574327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6846f-d20c-460b-ba8e-2ba9478e80f7 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:06.575003 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6846f-d20c-460b-ba8e-2ba9478e80f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:06.575304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6846f-d20c-460b-ba8e-2ba9478e80f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:06.576753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6846f-d20c-460b-ba8e-2ba9478e80f7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:06.578978 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7d491-a0cc-42b3-8ead-d08763235c23 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:06.590492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 15/29] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:06.596365 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6846f-d20c-460b-ba8e-2ba9478e80f7 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:06.640616 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:06.752040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df7d491-a0cc-42b3-8ead-d08763235c23 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:06.758064 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6846f-d20c-460b-ba8e-2ba9478e80f7 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:06.791126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fcf57-3f1a-4f99-bfc9-087b6513ba80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:06.791866 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fcf57-3f1a-4f99-bfc9-087b6513ba80 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:06.792222 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fcf57-3f1a-4f99-bfc9-087b6513ba80 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:06.793107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fcf57-3f1a-4f99-bfc9-087b6513ba80 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:06.794280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2fcf57-3f1a-4f99-bfc9-087b6513ba80 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:06.796491 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2fcf57-3f1a-4f99-bfc9-087b6513ba80 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:06.799459 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224b75f-98f8-4707-8e70-7ffc97969bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:06.800301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224b75f-98f8-4707-8e70-7ffc97969bfe None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:06.800777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224b75f-98f8-4707-8e70-7ffc97969bfe None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:06.801220 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224b75f-98f8-4707-8e70-7ffc97969bfe None admin] PATH_INFO: `/v3/users/7d770189a7bb4b81b8a42702f35fc84d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:06.802541 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224b75f-98f8-4707-8e70-7ffc97969bfe None admin] RBAC: Authorizing `identity:get_user(user_id=7d770189a7bb4b81b8a42702f35fc84d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:06.804670 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224b75f-98f8-4707-8e70-7ffc97969bfe None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:06.811848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 16/30] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:06 2022] GET /identity/v3/users/7d770189a7bb4b81b8a42702f35fc84d => generated 247 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:06.811848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f66db-67bb-4fd7-9eb4-b2db7e0262cd None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:06.811848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f66db-67bb-4fd7-9eb4-b2db7e0262cd None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:06.811848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f66db-67bb-4fd7-9eb4-b2db7e0262cd None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:06.811848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 17/31] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:06.812246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f66db-67bb-4fd7-9eb4-b2db7e0262cd None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:06.816711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:06.923867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f66db-67bb-4fd7-9eb4-b2db7e0262cd None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:06.934157 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 15/32] 10.222.0.60 () {60 vars in 1131 bytes} [Mon Sep 19 14:38:06 2022] POST /identity/v3/roles => generated 219 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:38:06.939878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fcf57-3f1a-4f99-bfc9-087b6513ba80 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:06.995135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420580b-1765-47c7-bbec-729232d98a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:06.996637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420580b-1765-47c7-bbec-729232d98a2d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:06.997078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420580b-1765-47c7-bbec-729232d98a2d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:06.997559 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420580b-1765-47c7-bbec-729232d98a2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:07.012770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420580b-1765-47c7-bbec-729232d98a2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:07.014604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0420580b-1765-47c7-bbec-729232d98a2d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-0420580b-1765-47c7-bbec-729232d98a2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:38:07.050210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:07.051327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:38:07.052294 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:38:07.052294 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:38:07.052294 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:07.054259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 18/33] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:07.055969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e4738-f4ff-4f9a-84b3-32a42e83b0a7 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:07.056477 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e4738-f4ff-4f9a-84b3-32a42e83b0a7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:07.056889 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e4738-f4ff-4f9a-84b3-32a42e83b0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:07.084531 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c16e4738-f4ff-4f9a-84b3-32a42e83b0a7 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:07.086061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c16e4738-f4ff-4f9a-84b3-32a42e83b0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:07.165673 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:07.173159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fcf57-3f1a-4f99-bfc9-087b6513ba80 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:07.177450 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:07.194020 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 19/34] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:07.196014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e4738-f4ff-4f9a-84b3-32a42e83b0a7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:07.215219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3b64f-559d-4e5f-887b-b0f0783ab92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:07.215996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3b64f-559d-4e5f-887b-b0f0783ab92d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:07.216283 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3b64f-559d-4e5f-887b-b0f0783ab92d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:07.216557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3b64f-559d-4e5f-887b-b0f0783ab92d None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:07.218034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3b64f-559d-4e5f-887b-b0f0783ab92d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:07.219795 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3b64f-559d-4e5f-887b-b0f0783ab92d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:07.222816 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 16/35] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:07.223902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d2415-b748-4003-b9c2-e066f0d65282 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:07.224193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d2415-b748-4003-b9c2-e066f0d65282 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:07.224506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d2415-b748-4003-b9c2-e066f0d65282 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:07.225910 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 17/36] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:07.229788 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d2415-b748-4003-b9c2-e066f0d65282 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:07.322742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e4738-f4ff-4f9a-84b3-32a42e83b0a7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:07.354525 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d2415-b748-4003-b9c2-e066f0d65282 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:07.373895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1c7a4-16d7-423b-b871-746e7979dd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:07.374521 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1c7a4-16d7-423b-b871-746e7979dd30 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:07.374804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1c7a4-16d7-423b-b871-746e7979dd30 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:07.375080 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1c7a4-16d7-423b-b871-746e7979dd30 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:07.376026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f1c7a4-16d7-423b-b871-746e7979dd30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:07.376336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f1c7a4-16d7-423b-b871-746e7979dd30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:07.378174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f1c7a4-16d7-423b-b871-746e7979dd30 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:07.394509 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f638a-3e7c-4421-a626-c1c2a5f16eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:07.395468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f638a-3e7c-4421-a626-c1c2a5f16eb5 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:07.396036 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f638a-3e7c-4421-a626-c1c2a5f16eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:07.396537 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f638a-3e7c-4421-a626-c1c2a5f16eb5 None admin] PATH_INFO: `/v3/domains/default/users/7d770189a7bb4b81b8a42702f35fc84d/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:07.399140 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f638a-3e7c-4421-a626-c1c2a5f16eb5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7d770189a7bb4b81b8a42702f35fc84d, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:07.399973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 20/37] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:07.401251 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f638a-3e7c-4421-a626-c1c2a5f16eb5 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:07.401357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd601fa-b379-403e-a451-88a9d87687fd None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:07.401649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd601fa-b379-403e-a451-88a9d87687fd None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:07.401959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd601fa-b379-403e-a451-88a9d87687fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:07.423640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 18/38] 10.222.0.60 () {58 vars in 1361 bytes} [Mon Sep 19 14:38:07 2022] PUT /identity/v3/domains/default/users/7d770189a7bb4b81b8a42702f35fc84d/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:38:07.425053 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-afd601fa-b379-403e-a451-88a9d87687fd None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:07.425170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f638a-3e7c-4421-a626-c1c2a5f16eb5 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:07.426549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-afd601fa-b379-403e-a451-88a9d87687fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:07.465525 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:07.496515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:07.546990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 21/39] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:07.582611 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9554e8-332e-4a8f-86ff-1a9db01fcf8f None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:07.582910 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9554e8-332e-4a8f-86ff-1a9db01fcf8f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:07.583192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9554e8-332e-4a8f-86ff-1a9db01fcf8f None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:07.591868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 22/40] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:38:07.592096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f638a-3e7c-4421-a626-c1c2a5f16eb5 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:07.596183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9554e8-332e-4a8f-86ff-1a9db01fcf8f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:07.642730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b36678-2a8f-4123-824d-cda0b319bcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:07.643372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b36678-2a8f-4123-824d-cda0b319bcca None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:07.643647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b36678-2a8f-4123-824d-cda0b319bcca None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:07.643958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b36678-2a8f-4123-824d-cda0b319bcca None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:07.644624 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b36678-2a8f-4123-824d-cda0b319bcca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:07.646101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b36678-2a8f-4123-824d-cda0b319bcca None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:07.679888 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 19/41] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Sep 19 14:38:07 2022] POST /identity/v3/projects => generated 301 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:38:07.681580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f17dd-b694-4eec-b195-303b17aaa000 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:07.681863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f17dd-b694-4eec-b195-303b17aaa000 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:07.682147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f17dd-b694-4eec-b195-303b17aaa000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:07.708452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f71f17dd-b694-4eec-b195-303b17aaa000 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:07.708452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f71f17dd-b694-4eec-b195-303b17aaa000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:07.775462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9554e8-332e-4a8f-86ff-1a9db01fcf8f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:07.807429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 20/42] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:07.867695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba1ca5-2ef2-4d24-afaf-326b677cfaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:07.868361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba1ca5-2ef2-4d24-afaf-326b677cfaac None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:07.868644 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba1ca5-2ef2-4d24-afaf-326b677cfaac None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:07.868915 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba1ca5-2ef2-4d24-afaf-326b677cfaac None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:07.869597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba1ca5-2ef2-4d24-afaf-326b677cfaac None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:07.871042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba1ca5-2ef2-4d24-afaf-326b677cfaac None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:07.932757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f17dd-b694-4eec-b195-303b17aaa000 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:07.942281 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:08.003451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 23/43] 10.222.0.60 () {60 vars in 1131 bytes} [Mon Sep 19 14:38:07 2022] POST /identity/v3/roles => generated 225 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:38:08.063414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f17dd-b694-4eec-b195-303b17aaa000 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:08.112544 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c248a-2b0e-47e5-ac94-168808ec370a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:08.113183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c248a-2b0e-47e5-ac94-168808ec370a None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:08.113464 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c248a-2b0e-47e5-ac94-168808ec370a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:08.113735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c248a-2b0e-47e5-ac94-168808ec370a None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:08.114437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c248a-2b0e-47e5-ac94-168808ec370a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:08.115900 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c248a-2b0e-47e5-ac94-168808ec370a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:08.134662 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 21/44] 10.222.0.60 () {60 vars in 1131 bytes} [Mon Sep 19 14:38:07 2022] POST /identity/v3/roles => generated 223 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:38:08.157970 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba1ca5-2ef2-4d24-afaf-326b677cfaac None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:08.318299 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:08.391496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba1ca5-2ef2-4d24-afaf-326b677cfaac None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:08.427253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50399a-d37f-4a12-80a4-1d9758cb9817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:08.427928 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50399a-d37f-4a12-80a4-1d9758cb9817 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:08.428220 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50399a-d37f-4a12-80a4-1d9758cb9817 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:08.428502 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50399a-d37f-4a12-80a4-1d9758cb9817 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:08.429716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50399a-d37f-4a12-80a4-1d9758cb9817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:08.431295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50399a-d37f-4a12-80a4-1d9758cb9817 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:08.432904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 24/45] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:08.438612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c248a-2b0e-47e5-ac94-168808ec370a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:08.506322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:08.542106 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c248a-2b0e-47e5-ac94-168808ec370a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:08.575655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de3c80-a9b4-40bf-91fc-f6fef96ad1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:08.576237 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de3c80-a9b4-40bf-91fc-f6fef96ad1e1 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:08.576521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de3c80-a9b4-40bf-91fc-f6fef96ad1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:08.576797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de3c80-a9b4-40bf-91fc-f6fef96ad1e1 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:08.577451 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de3c80-a9b4-40bf-91fc-f6fef96ad1e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:08.578903 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de3c80-a9b4-40bf-91fc-f6fef96ad1e1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:08.605387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 22/46] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Sep 19 14:38:08 2022] POST /identity/v3/projects => generated 312 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:38:09.732094 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a0a3c-c258-4836-b973-374e69f815d2 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:09.732694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a0a3c-c258-4836-b973-374e69f815d2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:09.732694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a0a3c-c258-4836-b973-374e69f815d2 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:09.734226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 25/47] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:09.739567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923330-4f75-46c4-8b0b-ffc1db0675c6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:09.740021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923330-4f75-46c4-8b0b-ffc1db0675c6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:09.740161 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923330-4f75-46c4-8b0b-ffc1db0675c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:09.758172 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b7923330-4f75-46c4-8b0b-ffc1db0675c6 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:09.759373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b7923330-4f75-46c4-8b0b-ffc1db0675c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:09.827084 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 23/48] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:10.092400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a0a3c-c258-4836-b973-374e69f815d2 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:10.199669 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a0a3c-c258-4836-b973-374e69f815d2 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:10.202047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:10.235206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05420b2-32a4-4643-87c0-9cb3ddc1e59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:10.235866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05420b2-32a4-4643-87c0-9cb3ddc1e59a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:10.236159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05420b2-32a4-4643-87c0-9cb3ddc1e59a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:10.236437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05420b2-32a4-4643-87c0-9cb3ddc1e59a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:10.246525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05420b2-32a4-4643-87c0-9cb3ddc1e59a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-d05420b2-32a4-4643-87c0-9cb3ddc1e59a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:10.251393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:10.252283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:10.253042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:10.253042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:10.253042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:10.253042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:10.253042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:10.253042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:10.253042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:10.253042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:10.253042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:10.253042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:10.253042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:10.253042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:10.253042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 26/49] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:38:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:10.258558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7923330-4f75-46c4-8b0b-ffc1db0675c6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:10.307435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:10.361912 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7923330-4f75-46c4-8b0b-ffc1db0675c6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:10.395347 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51bbab4-3dbd-4655-b9b0-55b184f4f1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:10.395993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bbab4-3dbd-4655-b9b0-55b184f4f1e9 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:10.396281 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bbab4-3dbd-4655-b9b0-55b184f4f1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:10.396559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bbab4-3dbd-4655-b9b0-55b184f4f1e9 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:10.397227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51bbab4-3dbd-4655-b9b0-55b184f4f1e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:10.397530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51bbab4-3dbd-4655-b9b0-55b184f4f1e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:10.398923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51bbab4-3dbd-4655-b9b0-55b184f4f1e9 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:10.412019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 24/50] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:10.418318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05420b2-32a4-4643-87c0-9cb3ddc1e59a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:10.522208 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05420b2-32a4-4643-87c0-9cb3ddc1e59a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:10.556107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b929f-b7e4-45f8-94f4-4f6343d5fff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:10.556745 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b929f-b7e4-45f8-94f4-4f6343d5fff3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:10.557030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b929f-b7e4-45f8-94f4-4f6343d5fff3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:10.557350 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b929f-b7e4-45f8-94f4-4f6343d5fff3 None admin] PATH_INFO: `/v3/users/7d770189a7bb4b81b8a42702f35fc84d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:10.558869 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b929f-b7e4-45f8-94f4-4f6343d5fff3 None admin] RBAC: Authorizing `identity:get_user(user_id=7d770189a7bb4b81b8a42702f35fc84d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:10.560529 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19b929f-b7e4-45f8-94f4-4f6343d5fff3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:10.562250 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 27/51] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:10 2022] GET /identity/v3/users/7d770189a7bb4b81b8a42702f35fc84d => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:10.567842 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51bbab4-3dbd-4655-b9b0-55b184f4f1e9 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:10.774787 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51bbab4-3dbd-4655-b9b0-55b184f4f1e9 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:10.785607 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:10.810153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3839766-498e-448f-b502-489e97c656b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:10.810801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3839766-498e-448f-b502-489e97c656b9 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:10.811086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3839766-498e-448f-b502-489e97c656b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:10.811364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3839766-498e-448f-b502-489e97c656b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:10.812478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3839766-498e-448f-b502-489e97c656b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:10.813924 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3839766-498e-448f-b502-489e97c656b9 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:10.815496 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 25/52] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:10.821418 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b929f-b7e4-45f8-94f4-4f6343d5fff3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:10.840843 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932482f9-a520-4cd9-a5db-6c26e571a454 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:10.841138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932482f9-a520-4cd9-a5db-6c26e571a454 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:10.841422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932482f9-a520-4cd9-a5db-6c26e571a454 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:10.842748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 26/53] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:38:10.848008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebb47c-f72c-40ce-9fb8-f16ede1ccea9 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:10.848300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebb47c-f72c-40ce-9fb8-f16ede1ccea9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:10.848579 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebb47c-f72c-40ce-9fb8-f16ede1ccea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:10.853544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:10.867604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5eebb47c-f72c-40ce-9fb8-f16ede1ccea9 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:10.868861 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5eebb47c-f72c-40ce-9fb8-f16ede1ccea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:10.927741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b929f-b7e4-45f8-94f4-4f6343d5fff3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:10.936705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 27/54] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:10.961666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c7835f-a03e-4d40-9af2-ebfa9a917d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:10.962265 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7835f-a03e-4d40-9af2-ebfa9a917d07 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:10.962547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7835f-a03e-4d40-9af2-ebfa9a917d07 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:10.962820 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7835f-a03e-4d40-9af2-ebfa9a917d07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:10.963531 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7835f-a03e-4d40-9af2-ebfa9a917d07 None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.domain.id=default, user.id=7d770189a7bb4b81b8a42702f35fc84d) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:10.963864 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7835f-a03e-4d40-9af2-ebfa9a917d07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:10.965343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7835f-a03e-4d40-9af2-ebfa9a917d07 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:10.971195 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 28/55] 10.222.0.60 () {56 vars in 1322 bytes} [Mon Sep 19 14:38:10 2022] GET /identity/v3/role_assignments?user.id=7d770189a7bb4b81b8a42702f35fc84d&scope.domain.id=default&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:11.218736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebb47c-f72c-40ce-9fb8-f16ede1ccea9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:11.314880 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebb47c-f72c-40ce-9fb8-f16ede1ccea9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:11.324362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:11.352540 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6715c-1882-4a33-b1e8-ef5dac323406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:11.353096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6715c-1882-4a33-b1e8-ef5dac323406 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:11.353383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6715c-1882-4a33-b1e8-ef5dac323406 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:11.353661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6715c-1882-4a33-b1e8-ef5dac323406 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:11.354766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6715c-1882-4a33-b1e8-ef5dac323406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:11.356803 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d6715c-1882-4a33-b1e8-ef5dac323406 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:11.358362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 28/56] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:11.364053 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c7835f-a03e-4d40-9af2-ebfa9a917d07 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:11.456010 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:11.462050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c7835f-a03e-4d40-9af2-ebfa9a917d07 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:11.495521 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cbf20-020f-4833-be6e-7a691d17ef4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:11.496217 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cbf20-020f-4833-be6e-7a691d17ef4d None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:11.496525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cbf20-020f-4833-be6e-7a691d17ef4d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:11.496800 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cbf20-020f-4833-be6e-7a691d17ef4d None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:11.497520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941cbf20-020f-4833-be6e-7a691d17ef4d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:11.499078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941cbf20-020f-4833-be6e-7a691d17ef4d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:11.529566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 29/57] 10.222.0.60 () {60 vars in 1140 bytes} [Mon Sep 19 14:38:11 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:38:13.691687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9b350-4894-4c2d-9ebf-0feef3cc1e7f None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:13.692249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9b350-4894-4c2d-9ebf-0feef3cc1e7f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:13.692249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9b350-4894-4c2d-9ebf-0feef3cc1e7f None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:13.693595 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 29/58] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:13.698911 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa830932-1d0e-40ed-9bea-028e76f9826f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:13.699901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa830932-1d0e-40ed-9bea-028e76f9826f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:13.700188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa830932-1d0e-40ed-9bea-028e76f9826f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:13.718177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-fa830932-1d0e-40ed-9bea-028e76f9826f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:13.719703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-fa830932-1d0e-40ed-9bea-028e76f9826f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:13.784358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 30/59] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:14.055201 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9b350-4894-4c2d-9ebf-0feef3cc1e7f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:14.154669 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9b350-4894-4c2d-9ebf-0feef3cc1e7f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:14.188348 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b5dfb-8b26-46ba-9352-52f1d116ed26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:14.188973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b5dfb-8b26-46ba-9352-52f1d116ed26 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:14.189256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b5dfb-8b26-46ba-9352-52f1d116ed26 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:14.189533 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b5dfb-8b26-46ba-9352-52f1d116ed26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:14.190650 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b5dfb-8b26-46ba-9352-52f1d116ed26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:14.192115 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b5dfb-8b26-46ba-9352-52f1d116ed26 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:14.193674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 30/60] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:14.199458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa830932-1d0e-40ed-9bea-028e76f9826f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:14.275319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:14.297776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa830932-1d0e-40ed-9bea-028e76f9826f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:14.330866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a04ca5-ff52-469d-b1ce-a3e137b9dcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:14.331411 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a04ca5-ff52-469d-b1ce-a3e137b9dcdc None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:14.331696 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a04ca5-ff52-469d-b1ce-a3e137b9dcdc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:14.332008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a04ca5-ff52-469d-b1ce-a3e137b9dcdc None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:14.332673 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a04ca5-ff52-469d-b1ce-a3e137b9dcdc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:14.334127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a04ca5-ff52-469d-b1ce-a3e137b9dcdc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:14.359887 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 31/61] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:38:14 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:38:17.642037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925648da-2c98-41a3-9da9-457e398fdecb None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:17.643069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925648da-2c98-41a3-9da9-457e398fdecb None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:17.643616 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925648da-2c98-41a3-9da9-457e398fdecb None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:17.645196 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 31/62] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:17.650754 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062325b-70a8-4490-a853-cc2498b66b38 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:17.651590 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062325b-70a8-4490-a853-cc2498b66b38 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:17.652093 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062325b-70a8-4490-a853-cc2498b66b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:17.677586 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9062325b-70a8-4490-a853-cc2498b66b38 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:17.679660 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9062325b-70a8-4490-a853-cc2498b66b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:17.777161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 32/63] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:17.872502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492335e7-66de-4f69-92db-b33cfe1116ec None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:17.873129 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492335e7-66de-4f69-92db-b33cfe1116ec None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:17.873626 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492335e7-66de-4f69-92db-b33cfe1116ec None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:17.875221 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 32/64] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:17.881575 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca89fa6-7ca2-4b7c-93b1-4bd11d9ca98a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:17.881861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca89fa6-7ca2-4b7c-93b1-4bd11d9ca98a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:17.882145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca89fa6-7ca2-4b7c-93b1-4bd11d9ca98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:17.913156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-eca89fa6-7ca2-4b7c-93b1-4bd11d9ca98a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:17.914412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-eca89fa6-7ca2-4b7c-93b1-4bd11d9ca98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:17.998428 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:18.007525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 33/65] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:18.033257 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3efe3-b400-4995-a176-bd8aaabd3296 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:18.033856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3efe3-b400-4995-a176-bd8aaabd3296 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:18.034371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3efe3-b400-4995-a176-bd8aaabd3296 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:18.035930 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 33/66] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:18.053384 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d94cb4-351e-40c8-bf4e-935331566f56 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:18.053676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d94cb4-351e-40c8-bf4e-935331566f56 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:18.053961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d94cb4-351e-40c8-bf4e-935331566f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:18.085432 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d8d94cb4-351e-40c8-bf4e-935331566f56 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:18.086088 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d8d94cb4-351e-40c8-bf4e-935331566f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:18.125103 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3efe3-b400-4995-a176-bd8aaabd3296 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:18.195311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 34/67] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:18.212444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:18.270349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3efe3-b400-4995-a176-bd8aaabd3296 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:18.299119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ae48f-0b04-4d2e-b908-a931e462ebc0 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:18.299772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ae48f-0b04-4d2e-b908-a931e462ebc0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:18.300281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ae48f-0b04-4d2e-b908-a931e462ebc0 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:18.301880 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 35/68] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:38:18.310013 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5978e40-dbc2-4881-a9f9-b5aab5b8f603 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:18.310627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5978e40-dbc2-4881-a9f9-b5aab5b8f603 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:18.311107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5978e40-dbc2-4881-a9f9-b5aab5b8f603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:18.331892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69dbad-b4af-48e1-b388-fa1fb26b62cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:18.332893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69dbad-b4af-48e1-b388-fa1fb26b62cb None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:18.333396 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69dbad-b4af-48e1-b388-fa1fb26b62cb None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:18.333866 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69dbad-b4af-48e1-b388-fa1fb26b62cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:18.341266 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f5978e40-dbc2-4881-a9f9-b5aab5b8f603 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:18.344917 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f5978e40-dbc2-4881-a9f9-b5aab5b8f603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:18.362515 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69dbad-b4af-48e1-b388-fa1fb26b62cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-9c69dbad-b4af-48e1-b388-fa1fb26b62cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:18.367844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:18.368679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:18.369469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:18.369469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:18.369469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:18.369469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:18.369469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:18.369469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:18.369469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:18.369469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:18.369469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:18.369469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:18.369469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:18.369469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:18.372321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 34/69] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:38:18.378529 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69dbad-b4af-48e1-b388-fa1fb26b62cb None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:18.442380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 36/70] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:18.443653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5978e40-dbc2-4881-a9f9-b5aab5b8f603 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:18.504592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69dbad-b4af-48e1-b388-fa1fb26b62cb None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:18.507981 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:18.509194 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:18.543954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c661d-abfa-4682-a8de-de969ad9f3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:18.545339 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c661d-abfa-4682-a8de-de969ad9f3c6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:18.545638 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c661d-abfa-4682-a8de-de969ad9f3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:18.545921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c661d-abfa-4682-a8de-de969ad9f3c6 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:18.546738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c661d-abfa-4682-a8de-de969ad9f3c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:18.547049 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c661d-abfa-4682-a8de-de969ad9f3c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:18.548471 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c661d-abfa-4682-a8de-de969ad9f3c6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:18.556693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 35/71] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:18.556693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896a39b-1aea-4877-bf47-49c239c18f92 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:18.556693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896a39b-1aea-4877-bf47-49c239c18f92 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:18.556693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896a39b-1aea-4877-bf47-49c239c18f92 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:18.556693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 36/72] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:18.556693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896a39b-1aea-4877-bf47-49c239c18f92 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:18.561741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5978e40-dbc2-4881-a9f9-b5aab5b8f603 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:18.600966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0044bf0-bb73-4ba1-b9af-a18a79cc8fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:18.601716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0044bf0-bb73-4ba1-b9af-a18a79cc8fda None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:18.602006 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0044bf0-bb73-4ba1-b9af-a18a79cc8fda None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:18.602394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0044bf0-bb73-4ba1-b9af-a18a79cc8fda None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:18.612940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0044bf0-bb73-4ba1-b9af-a18a79cc8fda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-d0044bf0-bb73-4ba1-b9af-a18a79cc8fda None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:18.618239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:18.619113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:18.619924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:18.619924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:18.619924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:18.619924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:18.619924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:18.619924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:18.619924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:18.619924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:18.619924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:18.619924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:18.619924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:38:18.619924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:18.619924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 37/73] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:38:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:18.621458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bfe94-60c5-403e-8b02-f29d26ee743a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:18.621558 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bfe94-60c5-403e-8b02-f29d26ee743a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:18.621776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bfe94-60c5-403e-8b02-f29d26ee743a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:18.639198 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e53bfe94-60c5-403e-8b02-f29d26ee743a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:18.640547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e53bfe94-60c5-403e-8b02-f29d26ee743a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:18.678303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896a39b-1aea-4877-bf47-49c239c18f92 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:18.709922 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 38/74] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:18.716878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c19110-e4ca-458e-90bc-27f17e030d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:18.717524 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c19110-e4ca-458e-90bc-27f17e030d1b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:18.717808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c19110-e4ca-458e-90bc-27f17e030d1b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:18.718084 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c19110-e4ca-458e-90bc-27f17e030d1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:18.720734 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bfe94-60c5-403e-8b02-f29d26ee743a None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:18.729090 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c19110-e4ca-458e-90bc-27f17e030d1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-73c19110-e4ca-458e-90bc-27f17e030d1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:18.734125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:18.735060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:18.735869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:18.735869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:18.735869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:18.735869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:18.735869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:18.735869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:18.735869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:18.735869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:18.735869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:18.735869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:18.735869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:18.735869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:18.735869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 37/75] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:38:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:18.736549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c19110-e4ca-458e-90bc-27f17e030d1b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:18.770261 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:18.784279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:18.848625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bfe94-60c5-403e-8b02-f29d26ee743a None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:18.850123 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c19110-e4ca-458e-90bc-27f17e030d1b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:18.886066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffce95-ce64-4da2-94e5-e66d1fe074c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:18.887319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffce95-ce64-4da2-94e5-e66d1fe074c7 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:18.887650 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffce95-ce64-4da2-94e5-e66d1fe074c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:18.887963 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffce95-ce64-4da2-94e5-e66d1fe074c7 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:18.888642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ffce95-ce64-4da2-94e5-e66d1fe074c7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:18.888949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ffce95-ce64-4da2-94e5-e66d1fe074c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:18.890349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ffce95-ce64-4da2-94e5-e66d1fe074c7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:18.892706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738f89a-9db3-4e79-b0cc-422d168e342b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:18.893430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738f89a-9db3-4e79-b0cc-422d168e342b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:18.893722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738f89a-9db3-4e79-b0cc-422d168e342b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:18.894002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738f89a-9db3-4e79-b0cc-422d168e342b None admin] PATH_INFO: `/v3/users/89fc893d73ba473db202a90d053d6fd3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:18.898909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 38/76] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:38:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:18.899725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffce95-ce64-4da2-94e5-e66d1fe074c7 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:18.913190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738f89a-9db3-4e79-b0cc-422d168e342b None admin] RBAC: Authorizing `identity:get_user(user_id=89fc893d73ba473db202a90d053d6fd3)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:18.914802 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738f89a-9db3-4e79-b0cc-422d168e342b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:18.921227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 39/77] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:18 2022] GET /identity/v3/users/89fc893d73ba473db202a90d053d6fd3 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:18.921227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738f89a-9db3-4e79-b0cc-422d168e342b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:19.001749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:19.015181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffce95-ce64-4da2-94e5-e66d1fe074c7 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:19.027945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738f89a-9db3-4e79-b0cc-422d168e342b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:19.029414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:19.049713 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806fd715-8edd-4eae-b2d7-d80dbc3dffc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:19.050335 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fd715-8edd-4eae-b2d7-d80dbc3dffc5 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:19.050617 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fd715-8edd-4eae-b2d7-d80dbc3dffc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:19.050913 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fd715-8edd-4eae-b2d7-d80dbc3dffc5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:19.061661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806fd715-8edd-4eae-b2d7-d80dbc3dffc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:19.063289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115d6f6-6e82-479f-9a46-5440f779d91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:19.063940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115d6f6-6e82-479f-9a46-5440f779d91f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:19.064230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115d6f6-6e82-479f-9a46-5440f779d91f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:19.064510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115d6f6-6e82-479f-9a46-5440f779d91f None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:19.065202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115d6f6-6e82-479f-9a46-5440f779d91f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:19.065509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115d6f6-6e82-479f-9a46-5440f779d91f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-806fd715-8edd-4eae-b2d7-d80dbc3dffc5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:19.066797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:19.067725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:19.068557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:19.068557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:19.068557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:19.068557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:19.068557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:19.068557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:19.068557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:19.068557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:19.068557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:19.068557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:19.068557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:19.068557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:19.069154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115d6f6-6e82-479f-9a46-5440f779d91f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:19.069220 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 39/78] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:19.069220 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806fd715-8edd-4eae-b2d7-d80dbc3dffc5 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:19.079963 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 40/79] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:19.080913 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115d6f6-6e82-479f-9a46-5440f779d91f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:19.178461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806fd715-8edd-4eae-b2d7-d80dbc3dffc5 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:19.186324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115d6f6-6e82-479f-9a46-5440f779d91f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:19.213311 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb17df-f628-4e80-823d-309a0c041341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:19.213953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb17df-f628-4e80-823d-309a0c041341 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:19.214239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb17df-f628-4e80-823d-309a0c041341 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:19.214519 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebb17df-f628-4e80-823d-309a0c041341 None admin] PATH_INFO: `/v3/users/89fc893d73ba473db202a90d053d6fd3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:19.216236 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb17df-f628-4e80-823d-309a0c041341 None admin] RBAC: Authorizing `identity:get_user(user_id=89fc893d73ba473db202a90d053d6fd3)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:19.217694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebb17df-f628-4e80-823d-309a0c041341 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:19.219323 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 40/80] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:19 2022] GET /identity/v3/users/89fc893d73ba473db202a90d053d6fd3 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:19.220129 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb17df-f628-4e80-823d-309a0c041341 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:19.222362 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3ff7f-b633-41b0-b9fa-18e3a0bedb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:19.223099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3ff7f-b633-41b0-b9fa-18e3a0bedb58 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:19.223382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3ff7f-b633-41b0-b9fa-18e3a0bedb58 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:19.224054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3ff7f-b633-41b0-b9fa-18e3a0bedb58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:19.246355 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:19.262416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:19.303011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3ff7f-b633-41b0-b9fa-18e3a0bedb58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:19.304473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3ff7f-b633-41b0-b9fa-18e3a0bedb58 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:19.334713 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebb17df-f628-4e80-823d-309a0c041341 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:19.369244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680456e9-a2e8-46dc-9662-bb0b97f25c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:19.369883 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680456e9-a2e8-46dc-9662-bb0b97f25c3a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:19.370176 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680456e9-a2e8-46dc-9662-bb0b97f25c3a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:19.370456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680456e9-a2e8-46dc-9662-bb0b97f25c3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:19.371652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680456e9-a2e8-46dc-9662-bb0b97f25c3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:19.373150 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680456e9-a2e8-46dc-9662-bb0b97f25c3a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:19.374699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 41/81] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:19.375523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680456e9-a2e8-46dc-9662-bb0b97f25c3a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:19.406675 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 41/82] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:38:19.409100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3ff7f-b633-41b0-b9fa-18e3a0bedb58 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:19.463302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:19.474114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:19.480451 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680456e9-a2e8-46dc-9662-bb0b97f25c3a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:19.511386 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3ff7f-b633-41b0-b9fa-18e3a0bedb58 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:19.513546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7848244-c31c-4303-8792-3a410034c687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:19.514156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7848244-c31c-4303-8792-3a410034c687 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:19.514443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7848244-c31c-4303-8792-3a410034c687 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:19.514717 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7848244-c31c-4303-8792-3a410034c687 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:19.515365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7848244-c31c-4303-8792-3a410034c687 None admin] RBAC: Adding query filter params (name=member) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:19.515666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7848244-c31c-4303-8792-3a410034c687 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:19.517068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7848244-c31c-4303-8792-3a410034c687 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:19.524917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 42/83] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:19.525693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7848244-c31c-4303-8792-3a410034c687 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:19.549643 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9dad0-f75d-4a70-a2a0-a0c013d4efd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:19.550296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9dad0-f75d-4a70-a2a0-a0c013d4efd4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:19.550568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9dad0-f75d-4a70-a2a0-a0c013d4efd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:19.550844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9dad0-f75d-4a70-a2a0-a0c013d4efd4 None admin] PATH_INFO: `/v3/users/7d770189a7bb4b81b8a42702f35fc84d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:19.551994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9dad0-f75d-4a70-a2a0-a0c013d4efd4 None admin] RBAC: Authorizing `identity:get_user(user_id=7d770189a7bb4b81b8a42702f35fc84d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:19.553493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9dad0-f75d-4a70-a2a0-a0c013d4efd4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:19.555171 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 42/84] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:19 2022] GET /identity/v3/users/7d770189a7bb4b81b8a42702f35fc84d => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:19.555848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9dad0-f75d-4a70-a2a0-a0c013d4efd4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:19.631574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7848244-c31c-4303-8792-3a410034c687 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:19.655856 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9dad0-f75d-4a70-a2a0-a0c013d4efd4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:19.665564 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b762b81-21ad-4c1c-92ba-d4c9fcfdc7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:19.666188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b762b81-21ad-4c1c-92ba-d4c9fcfdc7fd None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:19.666478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b762b81-21ad-4c1c-92ba-d4c9fcfdc7fd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:19.666760 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b762b81-21ad-4c1c-92ba-d4c9fcfdc7fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:19.683142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:19.686657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:19.692264 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33008c16-41a9-4907-b217-0f4fae7493b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:19.692870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33008c16-41a9-4907-b217-0f4fae7493b4 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:19.693150 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33008c16-41a9-4907-b217-0f4fae7493b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:19.693429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33008c16-41a9-4907-b217-0f4fae7493b4 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:19.694080 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33008c16-41a9-4907-b217-0f4fae7493b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:19.695525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33008c16-41a9-4907-b217-0f4fae7493b4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:19.715748 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 43/85] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:38:19 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:38:19.716595 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33008c16-41a9-4907-b217-0f4fae7493b4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:19.753460 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b762b81-21ad-4c1c-92ba-d4c9fcfdc7fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:19.755214 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b762b81-21ad-4c1c-92ba-d4c9fcfdc7fd None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:19.847161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33008c16-41a9-4907-b217-0f4fae7493b4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:19.868648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 43/86] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:38:19.872965 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b762b81-21ad-4c1c-92ba-d4c9fcfdc7fd None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:19.891852 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0a62a-ef78-4a05-af76-2be3f68777a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:19.892521 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0a62a-ef78-4a05-af76-2be3f68777a2 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:19.892809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0a62a-ef78-4a05-af76-2be3f68777a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:19.893094 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0a62a-ef78-4a05-af76-2be3f68777a2 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:19.894291 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0a62a-ef78-4a05-af76-2be3f68777a2 None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:19.895729 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0a62a-ef78-4a05-af76-2be3f68777a2 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:19.898337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 44/87] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:19 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:19.898337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0a62a-ef78-4a05-af76-2be3f68777a2 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:19.937138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:19.963263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:20.006371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b762b81-21ad-4c1c-92ba-d4c9fcfdc7fd None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:20.017316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0a62a-ef78-4a05-af76-2be3f68777a2 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:20.052085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d915d-74d2-42f1-8aa9-dd15b6e139f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:20.052728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d915d-74d2-42f1-8aa9-dd15b6e139f8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:20.053012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d915d-74d2-42f1-8aa9-dd15b6e139f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:20.053313 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d915d-74d2-42f1-8aa9-dd15b6e139f8 None admin] PATH_INFO: `/v3/users/89fc893d73ba473db202a90d053d6fd3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:20.054448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d915d-74d2-42f1-8aa9-dd15b6e139f8 None admin] RBAC: Authorizing `identity:get_user(user_id=89fc893d73ba473db202a90d053d6fd3)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:20.055946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d915d-74d2-42f1-8aa9-dd15b6e139f8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:20.057529 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 44/88] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:19 2022] GET /identity/v3/users/89fc893d73ba473db202a90d053d6fd3 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:20.058294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d915d-74d2-42f1-8aa9-dd15b6e139f8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:20.058874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549dbcaf-1d42-4d1e-bbbf-fdd30b6610ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:20.059506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dbcaf-1d42-4d1e-bbbf-fdd30b6610ec None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:20.059980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dbcaf-1d42-4d1e-bbbf-fdd30b6610ec None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:20.060324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dbcaf-1d42-4d1e-bbbf-fdd30b6610ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:20.147737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549dbcaf-1d42-4d1e-bbbf-fdd30b6610ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:20.149333 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549dbcaf-1d42-4d1e-bbbf-fdd30b6610ec None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:20.188079 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d915d-74d2-42f1-8aa9-dd15b6e139f8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:20.207046 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:20.209791 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:20.233823 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb70520-7bc7-4698-8148-072ba10fd4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:20.234565 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb70520-7bc7-4698-8148-072ba10fd4e7 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:20.234915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb70520-7bc7-4698-8148-072ba10fd4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:20.235266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb70520-7bc7-4698-8148-072ba10fd4e7 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:20.236501 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb70520-7bc7-4698-8148-072ba10fd4e7 None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:20.238467 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb70520-7bc7-4698-8148-072ba10fd4e7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:20.240685 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 45/89] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:20 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:20.242008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb70520-7bc7-4698-8148-072ba10fd4e7 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:20.273667 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 45/90] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:38:20.276583 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549dbcaf-1d42-4d1e-bbbf-fdd30b6610ec None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:20.375512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb70520-7bc7-4698-8148-072ba10fd4e7 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:20.413504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab119f49-be17-4fe1-8a37-980f594d441d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:20.414038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab119f49-be17-4fe1-8a37-980f594d441d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:20.414316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab119f49-be17-4fe1-8a37-980f594d441d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:20.414547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549dbcaf-1d42-4d1e-bbbf-fdd30b6610ec None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:20.414650 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab119f49-be17-4fe1-8a37-980f594d441d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:20.415300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab119f49-be17-4fe1-8a37-980f594d441d None admin] RBAC: Adding query filter params (role.id=0a6592b9a0b64df6938aba0a010c2256, scope.project.id=4681da4f1deb408293b2c948aec851ea, user.id=89fc893d73ba473db202a90d053d6fd3) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:20.415608 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab119f49-be17-4fe1-8a37-980f594d441d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:20.417104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab119f49-be17-4fe1-8a37-980f594d441d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:20.427493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 46/91] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:20 2022] GET /identity/v3/role_assignments?user.id=89fc893d73ba473db202a90d053d6fd3&scope.project.id=4681da4f1deb408293b2c948aec851ea&role.id=0a6592b9a0b64df6938aba0a010c2256 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:20.428594 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab119f49-be17-4fe1-8a37-980f594d441d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:20.491198 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8bfc8-36db-4308-bbce-c966cb025e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:20.491922 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8bfc8-36db-4308-bbce-c966cb025e1c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:20.492211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8bfc8-36db-4308-bbce-c966cb025e1c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:20.492489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8bfc8-36db-4308-bbce-c966cb025e1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:20.496789 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:20.548539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:20.626129 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab119f49-be17-4fe1-8a37-980f594d441d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:20.643467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8bfc8-36db-4308-bbce-c966cb025e1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:20.645778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8bfc8-36db-4308-bbce-c966cb025e1c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:20.733225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1cabc-7772-48ae-a276-2c667c84ebe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:20.733868 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1cabc-7772-48ae-a276-2c667c84ebe0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:20.734036 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1cabc-7772-48ae-a276-2c667c84ebe0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:20.734315 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1cabc-7772-48ae-a276-2c667c84ebe0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:20.735460 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1cabc-7772-48ae-a276-2c667c84ebe0 None admin] RBAC: Adding query filter params (role.id=fe3bbad209634f16b1fcb66d6d49f665, scope.project.id=4681da4f1deb408293b2c948aec851ea, user.id=89fc893d73ba473db202a90d053d6fd3) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:20.735460 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1cabc-7772-48ae-a276-2c667c84ebe0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:20.736872 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1cabc-7772-48ae-a276-2c667c84ebe0 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:20.758218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 47/92] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:20 2022] GET /identity/v3/role_assignments?user.id=89fc893d73ba473db202a90d053d6fd3&scope.project.id=4681da4f1deb408293b2c948aec851ea&role.id=fe3bbad209634f16b1fcb66d6d49f665 => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:20.759198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1cabc-7772-48ae-a276-2c667c84ebe0 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:20.767072 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 46/93] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:38:20.775363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8bfc8-36db-4308-bbce-c966cb025e1c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:20.876121 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:20.951108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8bfc8-36db-4308-bbce-c966cb025e1c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:21.018571 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb991b-6a3e-43ca-a663-b5547485ec98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:21.019182 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb991b-6a3e-43ca-a663-b5547485ec98 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:21.019471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb991b-6a3e-43ca-a663-b5547485ec98 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:21.019752 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb991b-6a3e-43ca-a663-b5547485ec98 None admin] PATH_INFO: `/v3/projects/950c54d4e4e44452809264618dea23df` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:21.024481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb991b-6a3e-43ca-a663-b5547485ec98 None admin] RBAC: Authorizing `identity:get_project(project_id=950c54d4e4e44452809264618dea23df)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:21.026222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb991b-6a3e-43ca-a663-b5547485ec98 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:21.028556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 47/94] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:20 2022] GET /identity/v3/projects/950c54d4e4e44452809264618dea23df => generated 312 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:21.034333 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb991b-6a3e-43ca-a663-b5547485ec98 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:21.085588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1cabc-7772-48ae-a276-2c667c84ebe0 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:21.114815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:21.196563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef992041-6e9f-49a5-b3d4-30c9a2e35855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:21.197225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef992041-6e9f-49a5-b3d4-30c9a2e35855 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:21.197512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef992041-6e9f-49a5-b3d4-30c9a2e35855 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:21.197788 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef992041-6e9f-49a5-b3d4-30c9a2e35855 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:21.204703 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef992041-6e9f-49a5-b3d4-30c9a2e35855 None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:21.206206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef992041-6e9f-49a5-b3d4-30c9a2e35855 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:21.207982 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 48/95] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:20 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:21.209878 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb991b-6a3e-43ca-a663-b5547485ec98 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:21.215870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef992041-6e9f-49a5-b3d4-30c9a2e35855 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:21.265962 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:21.297663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a89484-d1ca-45df-8ea0-a6fdd8a2ddcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:21.298527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a89484-d1ca-45df-8ea0-a6fdd8a2ddcc None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:21.298892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a89484-d1ca-45df-8ea0-a6fdd8a2ddcc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:21.299226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a89484-d1ca-45df-8ea0-a6fdd8a2ddcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:21.300129 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a89484-d1ca-45df-8ea0-a6fdd8a2ddcc None admin] RBAC: Adding query filter params (role.id=0a6592b9a0b64df6938aba0a010c2256, scope.project.id=950c54d4e4e44452809264618dea23df, user.id=89fc893d73ba473db202a90d053d6fd3) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:21.300469 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a89484-d1ca-45df-8ea0-a6fdd8a2ddcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:21.302335 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a89484-d1ca-45df-8ea0-a6fdd8a2ddcc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:21.323253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 48/96] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:21 2022] GET /identity/v3/role_assignments?user.id=89fc893d73ba473db202a90d053d6fd3&scope.project.id=950c54d4e4e44452809264618dea23df&role.id=0a6592b9a0b64df6938aba0a010c2256 => generated 259 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:21.404803 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef992041-6e9f-49a5-b3d4-30c9a2e35855 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:21.447592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74840d7-5411-436c-b85b-6efe51fe2c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:21.448310 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74840d7-5411-436c-b85b-6efe51fe2c47 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:21.448597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74840d7-5411-436c-b85b-6efe51fe2c47 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:21.448875 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74840d7-5411-436c-b85b-6efe51fe2c47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:21.450338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74840d7-5411-436c-b85b-6efe51fe2c47 None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=4681da4f1deb408293b2c948aec851ea, user.id=7d770189a7bb4b81b8a42702f35fc84d) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:21.450681 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74840d7-5411-436c-b85b-6efe51fe2c47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:21.452130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74840d7-5411-436c-b85b-6efe51fe2c47 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:21.464380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 49/97] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:21 2022] GET /identity/v3/role_assignments?user.id=7d770189a7bb4b81b8a42702f35fc84d&scope.project.id=4681da4f1deb408293b2c948aec851ea&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:22.285157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5092d94-834d-411f-8603-85af86518aeb None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:22.286157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5092d94-834d-411f-8603-85af86518aeb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:22.286696 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5092d94-834d-411f-8603-85af86518aeb None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:22.288268 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 49/98] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:22.304613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f269d-c095-41ff-af24-1942e9bf02fa None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:22.305587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f269d-c095-41ff-af24-1942e9bf02fa None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:22.306059 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f269d-c095-41ff-af24-1942e9bf02fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:22.332872 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b38f269d-c095-41ff-af24-1942e9bf02fa None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:22.334405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b38f269d-c095-41ff-af24-1942e9bf02fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:22.382030 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:22.422704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 50/99] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:22.974315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5092d94-834d-411f-8603-85af86518aeb None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:23.050578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c460f-fb82-47d4-ad48-05e6e2d66c75 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:23.050859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c460f-fb82-47d4-ad48-05e6e2d66c75 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:23.051296 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c460f-fb82-47d4-ad48-05e6e2d66c75 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:23.052939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 51/100] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:38:23.066372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80366051-52a4-4594-9eff-ec6cc8927fa0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:23.067179 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80366051-52a4-4594-9eff-ec6cc8927fa0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:23.067796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80366051-52a4-4594-9eff-ec6cc8927fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:23.096498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5092d94-834d-411f-8603-85af86518aeb None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:23.116833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-80366051-52a4-4594-9eff-ec6cc8927fa0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:23.118767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-80366051-52a4-4594-9eff-ec6cc8927fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:23.159685 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745eb47-0d11-41c6-b38b-eaf571ed546b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:23.160349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745eb47-0d11-41c6-b38b-eaf571ed546b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:23.160627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745eb47-0d11-41c6-b38b-eaf571ed546b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:23.160904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745eb47-0d11-41c6-b38b-eaf571ed546b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:23.178932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745eb47-0d11-41c6-b38b-eaf571ed546b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c745eb47-0d11-41c6-b38b-eaf571ed546b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:23.185245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:23.186133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:23.187305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:23.187305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:23.187305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:23.187305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:23.187305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:23.187305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:23.187305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:23.187305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:23.187305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:23.187305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:23.187305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:23.187305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:23.187305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 50/101] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:23.196005 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745eb47-0d11-41c6-b38b-eaf571ed546b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:23.243538 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 52/102] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 19 14:38:23.276207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:23.340363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745eb47-0d11-41c6-b38b-eaf571ed546b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:23.389645 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05e5fa-e05e-4e25-9575-4c5bc0880734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:23.390281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05e5fa-e05e-4e25-9575-4c5bc0880734 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:23.390563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05e5fa-e05e-4e25-9575-4c5bc0880734 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:23.390836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05e5fa-e05e-4e25-9575-4c5bc0880734 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:23.391508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05e5fa-e05e-4e25-9575-4c5bc0880734 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:23.391809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05e5fa-e05e-4e25-9575-4c5bc0880734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:23.393252 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05e5fa-e05e-4e25-9575-4c5bc0880734 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:23.408036 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 51/103] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:23.413789 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80366051-52a4-4594-9eff-ec6cc8927fa0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:23.531980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80366051-52a4-4594-9eff-ec6cc8927fa0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:23.533519 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:23.586098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b594a0c-83c3-491c-a182-153ae75670e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:23.586761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b594a0c-83c3-491c-a182-153ae75670e3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:23.587045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b594a0c-83c3-491c-a182-153ae75670e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:23.587320 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b594a0c-83c3-491c-a182-153ae75670e3 None admin] PATH_INFO: `/v3/users/c2dcd642c2e24ebdacea43238f322e82` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:23.609949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b594a0c-83c3-491c-a182-153ae75670e3 None admin] RBAC: Authorizing `identity:get_user(user_id=c2dcd642c2e24ebdacea43238f322e82)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:23.611461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b594a0c-83c3-491c-a182-153ae75670e3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:23.613469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 53/104] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:23 2022] GET /identity/v3/users/c2dcd642c2e24ebdacea43238f322e82 => generated 289 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:23.619418 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05e5fa-e05e-4e25-9575-4c5bc0880734 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:23.717686 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f335b-6cce-4130-8a6c-8cc7c38de324 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:23.717977 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f335b-6cce-4130-8a6c-8cc7c38de324 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:23.718255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f335b-6cce-4130-8a6c-8cc7c38de324 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:23.719808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 54/105] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:38:23.725521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0b2fe-b1be-4aaf-b41a-0ce109b1f52b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:23.726172 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0b2fe-b1be-4aaf-b41a-0ce109b1f52b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:23.726630 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0b2fe-b1be-4aaf-b41a-0ce109b1f52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:23.752477 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05e5fa-e05e-4e25-9575-4c5bc0880734 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:23.760976 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-fac0b2fe-b1be-4aaf-b41a-0ce109b1f52b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:23.763011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fac0b2fe-b1be-4aaf-b41a-0ce109b1f52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:23.768623 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:23.792933 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91edbdd-7dcd-4d87-99b5-ec0fceb7b283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:23.795194 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91edbdd-7dcd-4d87-99b5-ec0fceb7b283 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:23.795766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91edbdd-7dcd-4d87-99b5-ec0fceb7b283 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:23.796361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91edbdd-7dcd-4d87-99b5-ec0fceb7b283 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:23.895898 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 55/106] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:23.896773 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0b2fe-b1be-4aaf-b41a-0ce109b1f52b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:23.909123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91edbdd-7dcd-4d87-99b5-ec0fceb7b283 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:23.910606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91edbdd-7dcd-4d87-99b5-ec0fceb7b283 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:24.027923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:24.051730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0b2fe-b1be-4aaf-b41a-0ce109b1f52b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:24.082197 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 52/107] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:38:24.085303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbaf32-be30-4732-890c-584f4f50473b None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:24.085774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbaf32-be30-4732-890c-584f4f50473b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:24.086230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fbaf32-be30-4732-890c-584f4f50473b None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:24.088555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 53/108] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:24.093570 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879b23c-8b73-48e0-ab1c-70b928fa96da None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:24.093866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879b23c-8b73-48e0-ab1c-70b928fa96da None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:24.094153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879b23c-8b73-48e0-ab1c-70b928fa96da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:24.109851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac4877-cb65-4a73-99d7-0e6f63fe78d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:24.110497 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac4877-cb65-4a73-99d7-0e6f63fe78d8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:24.110779 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac4877-cb65-4a73-99d7-0e6f63fe78d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:24.111051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac4877-cb65-4a73-99d7-0e6f63fe78d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:24.112197 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac4877-cb65-4a73-99d7-0e6f63fe78d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:24.113651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ac4877-cb65-4a73-99d7-0e6f63fe78d8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:24.115239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 56/109] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:24.116034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac4877-cb65-4a73-99d7-0e6f63fe78d8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:24.136940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-4879b23c-8b73-48e0-ab1c-70b928fa96da None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:24.138164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4879b23c-8b73-48e0-ab1c-70b928fa96da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:24.156499 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:24.228760 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac4877-cb65-4a73-99d7-0e6f63fe78d8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:24.265775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c975f-e6a8-4816-8f0d-1fd7ec05ac84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:24.266704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c975f-e6a8-4816-8f0d-1fd7ec05ac84 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:24.267202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c975f-e6a8-4816-8f0d-1fd7ec05ac84 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:24.267746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c975f-e6a8-4816-8f0d-1fd7ec05ac84 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:24.268522 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 54/110] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:24.269617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879b23c-8b73-48e0-ab1c-70b928fa96da None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:24.272549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c975f-e6a8-4816-8f0d-1fd7ec05ac84 None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:24.274171 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c975f-e6a8-4816-8f0d-1fd7ec05ac84 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:24.276295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 57/111] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:24 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:24.277424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c975f-e6a8-4816-8f0d-1fd7ec05ac84 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:24.322801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:24.386635 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879b23c-8b73-48e0-ab1c-70b928fa96da None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:24.451871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137f18a-a13b-43d4-9cba-8032c73c365e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:24.452499 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137f18a-a13b-43d4-9cba-8032c73c365e None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:24.454917 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137f18a-a13b-43d4-9cba-8032c73c365e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:24.455200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137f18a-a13b-43d4-9cba-8032c73c365e None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:24.455869 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137f18a-a13b-43d4-9cba-8032c73c365e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:24.457343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137f18a-a13b-43d4-9cba-8032c73c365e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:24.478692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:24.504156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c975f-e6a8-4816-8f0d-1fd7ec05ac84 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:24.509911 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 55/112] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Sep 19 14:38:24 2022] POST /identity/v3/projects => generated 302 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:38:24.511025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137f18a-a13b-43d4-9cba-8032c73c365e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:24.584267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69990bf4-9f13-410c-beb7-b0b1cba9234a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:24.586480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69990bf4-9f13-410c-beb7-b0b1cba9234a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:24.588663 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69990bf4-9f13-410c-beb7-b0b1cba9234a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:24.589225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69990bf4-9f13-410c-beb7-b0b1cba9234a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:24.618742 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69990bf4-9f13-410c-beb7-b0b1cba9234a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-69990bf4-9f13-410c-beb7-b0b1cba9234a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:24.624280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:24.625234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:24.626034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:24.626034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:24.626034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:24.626034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:24.626034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:24.626034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:24.626034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:24.626034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:24.626034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:24.626034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:24.626034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:38:24.626034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:24.637446 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 58/113] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:38:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:24.643069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69990bf4-9f13-410c-beb7-b0b1cba9234a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:24.718364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137f18a-a13b-43d4-9cba-8032c73c365e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:24.760862 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539be08-6b42-4198-bc04-ea2e12a35711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:24.761494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539be08-6b42-4198-bc04-ea2e12a35711 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:24.761780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539be08-6b42-4198-bc04-ea2e12a35711 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:24.762579 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1539be08-6b42-4198-bc04-ea2e12a35711 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:24.763321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539be08-6b42-4198-bc04-ea2e12a35711 None admin] RBAC: Adding query filter params (role.id=2235d4074114441ea4661c3fb936dcc3, scope.project.id=4681da4f1deb408293b2c948aec851ea, user.id=c2dcd642c2e24ebdacea43238f322e82) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:24.763629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539be08-6b42-4198-bc04-ea2e12a35711 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:24.766094 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1539be08-6b42-4198-bc04-ea2e12a35711 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:24.774722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 56/114] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:24 2022] GET /identity/v3/role_assignments?user.id=c2dcd642c2e24ebdacea43238f322e82&scope.project.id=4681da4f1deb408293b2c948aec851ea&role.id=2235d4074114441ea4661c3fb936dcc3 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:24.776100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539be08-6b42-4198-bc04-ea2e12a35711 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:24.820857 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:24.848931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69990bf4-9f13-410c-beb7-b0b1cba9234a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:24.879619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:24.898009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257883d0-3aa5-4e31-874d-fa20a525cfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:24.898793 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257883d0-3aa5-4e31-874d-fa20a525cfc0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:24.899144 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257883d0-3aa5-4e31-874d-fa20a525cfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:24.900167 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257883d0-3aa5-4e31-874d-fa20a525cfc0 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:24.900167 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257883d0-3aa5-4e31-874d-fa20a525cfc0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:24.900441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257883d0-3aa5-4e31-874d-fa20a525cfc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:24.903204 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257883d0-3aa5-4e31-874d-fa20a525cfc0 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:24.913755 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 59/115] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:38:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:24.914531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353135c-6ab9-42b5-b05d-f6acf8e91193 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:24.914819 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353135c-6ab9-42b5-b05d-f6acf8e91193 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:24.915093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353135c-6ab9-42b5-b05d-f6acf8e91193 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:24.917466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 60/116] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:24.918346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc0b69-b16c-485e-b1dd-8a17ebeb8de8 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:24.918631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc0b69-b16c-485e-b1dd-8a17ebeb8de8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:24.918909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc0b69-b16c-485e-b1dd-8a17ebeb8de8 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:24.921125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 61/117] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:24.921125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc0b69-b16c-485e-b1dd-8a17ebeb8de8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:24.944467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1539be08-6b42-4198-bc04-ea2e12a35711 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:24.997027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa6f1c-3f84-4957-b5d3-8a5702fb5a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:24.997675 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6f1c-3f84-4957-b5d3-8a5702fb5a04 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:24.997955 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6f1c-3f84-4957-b5d3-8a5702fb5a04 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:24.998229 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa6f1c-3f84-4957-b5d3-8a5702fb5a04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:25.016818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa6f1c-3f84-4957-b5d3-8a5702fb5a04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-9cfa6f1c-3f84-4957-b5d3-8a5702fb5a04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:25.024330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:25.025240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:25.026133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:25.026133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:25.026133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:25.026133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:25.026133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:25.026133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:25.026133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:25.026133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:25.026133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:25.026133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:25.026133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:25.026133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:25.029387 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 57/118] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:25.034939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5dfe0-6d35-4ba7-9b68-fb5c52af6529 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:25.034939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5dfe0-6d35-4ba7-9b68-fb5c52af6529 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:25.034939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5dfe0-6d35-4ba7-9b68-fb5c52af6529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:25.052834 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-81f5dfe0-6d35-4ba7-9b68-fb5c52af6529 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:25.054036 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-81f5dfe0-6d35-4ba7-9b68-fb5c52af6529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:25.060361 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc0b69-b16c-485e-b1dd-8a17ebeb8de8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:25.105422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc82635-6f08-4a87-a4e9-64aa00bbff91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:25.106445 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc82635-6f08-4a87-a4e9-64aa00bbff91 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:25.106928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc82635-6f08-4a87-a4e9-64aa00bbff91 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:25.107448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc82635-6f08-4a87-a4e9-64aa00bbff91 None admin] PATH_INFO: `/v3/users/89fc893d73ba473db202a90d053d6fd3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:25.109509 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc82635-6f08-4a87-a4e9-64aa00bbff91 None admin] RBAC: Authorizing `identity:get_user(user_id=89fc893d73ba473db202a90d053d6fd3)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:25.112084 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc82635-6f08-4a87-a4e9-64aa00bbff91 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:25.114178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 62/119] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:24 2022] GET /identity/v3/users/89fc893d73ba473db202a90d053d6fd3 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:25.115518 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1abb87-bd97-4271-b945-e5baba7f7d94 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:25.116028 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1abb87-bd97-4271-b945-e5baba7f7d94 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:25.116669 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1abb87-bd97-4271-b945-e5baba7f7d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:25.143779 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-7d1abb87-bd97-4271-b945-e5baba7f7d94 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:25.144343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 58/120] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:25.145407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5dfe0-6d35-4ba7-9b68-fb5c52af6529 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:25.145755 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7d1abb87-bd97-4271-b945-e5baba7f7d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:25.161274 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:25.220678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:25.267631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 63/121] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:25.268700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1abb87-bd97-4271-b945-e5baba7f7d94 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:25.298355 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5dfe0-6d35-4ba7-9b68-fb5c52af6529 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:25.351451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b951d2-1874-4dee-a021-363dfe9204d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:25.352113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b951d2-1874-4dee-a021-363dfe9204d7 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:25.352398 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b951d2-1874-4dee-a021-363dfe9204d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:25.352674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b951d2-1874-4dee-a021-363dfe9204d7 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:25.354600 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b951d2-1874-4dee-a021-363dfe9204d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:25.354919 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b951d2-1874-4dee-a021-363dfe9204d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:25.357098 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b951d2-1874-4dee-a021-363dfe9204d7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:25.376059 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 59/122] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:25.381622 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b951d2-1874-4dee-a021-363dfe9204d7 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:25.445017 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1abb87-bd97-4271-b945-e5baba7f7d94 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:25.491500 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf685a9-5de0-49c8-9343-26057e8359e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:25.492366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf685a9-5de0-49c8-9343-26057e8359e1 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:25.492642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf685a9-5de0-49c8-9343-26057e8359e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:25.492921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf685a9-5de0-49c8-9343-26057e8359e1 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:25.494064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf685a9-5de0-49c8-9343-26057e8359e1 None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:25.495503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf685a9-5de0-49c8-9343-26057e8359e1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:25.497207 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 64/123] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:25 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:25.506249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf685a9-5de0-49c8-9343-26057e8359e1 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:25.547287 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:25.581840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b951d2-1874-4dee-a021-363dfe9204d7 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:25.603473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:25.654439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d97e3-5b2b-44cb-8924-c3726cbabbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:25.655084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d97e3-5b2b-44cb-8924-c3726cbabbda None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:25.655368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d97e3-5b2b-44cb-8924-c3726cbabbda None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:25.655640 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d97e3-5b2b-44cb-8924-c3726cbabbda None admin] PATH_INFO: `/v3/users/89fc893d73ba473db202a90d053d6fd3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:25.658176 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d97e3-5b2b-44cb-8924-c3726cbabbda None admin] RBAC: Authorizing `identity:get_user(user_id=89fc893d73ba473db202a90d053d6fd3)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:25.659655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37d97e3-5b2b-44cb-8924-c3726cbabbda None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:25.661267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 60/124] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:25 2022] GET /identity/v3/users/89fc893d73ba473db202a90d053d6fd3 => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:25.671157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d97e3-5b2b-44cb-8924-c3726cbabbda None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:25.712539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf685a9-5de0-49c8-9343-26057e8359e1 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:25.754489 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e9c61-d94c-4339-9ce0-10c5b91bf366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:25.755178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e9c61-d94c-4339-9ce0-10c5b91bf366 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:25.755458 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e9c61-d94c-4339-9ce0-10c5b91bf366 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:25.756071 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e9c61-d94c-4339-9ce0-10c5b91bf366 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea/users/89fc893d73ba473db202a90d053d6fd3/roles/fe3bbad209634f16b1fcb66d6d49f665` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:25.760666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e9c61-d94c-4339-9ce0-10c5b91bf366 None admin] RBAC: Authorizing `identity:create_grant(project_id=4681da4f1deb408293b2c948aec851ea, user_id=89fc893d73ba473db202a90d053d6fd3, role_id=fe3bbad209634f16b1fcb66d6d49f665)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:25.762280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e9c61-d94c-4339-9ce0-10c5b91bf366 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:25.786105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 65/125] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:38:25 2022] PUT /identity/v3/projects/4681da4f1deb408293b2c948aec851ea/users/89fc893d73ba473db202a90d053d6fd3/roles/fe3bbad209634f16b1fcb66d6d49f665 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:38:25.786105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e9c61-d94c-4339-9ce0-10c5b91bf366 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:25.854261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d97e3-5b2b-44cb-8924-c3726cbabbda None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:25.886628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:25.916047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e9c61-d94c-4339-9ce0-10c5b91bf366 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:25.950513 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b43192-08c3-4802-9575-6466a0f05bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:25.951520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b43192-08c3-4802-9575-6466a0f05bd1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:25.952035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b43192-08c3-4802-9575-6466a0f05bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:25.952505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b43192-08c3-4802-9575-6466a0f05bd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:25.961007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1a791-968c-49ca-9317-f0ff9b040725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:25.962065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1a791-968c-49ca-9317-f0ff9b040725 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:25.962348 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1a791-968c-49ca-9317-f0ff9b040725 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:25.962625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b1a791-968c-49ca-9317-f0ff9b040725 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:25.963766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1a791-968c-49ca-9317-f0ff9b040725 None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:25.967247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b1a791-968c-49ca-9317-f0ff9b040725 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:25.967381 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 66/126] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:25 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:25.968390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1a791-968c-49ca-9317-f0ff9b040725 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:25.977391 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b43192-08c3-4802-9575-6466a0f05bd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-e0b43192-08c3-4802-9575-6466a0f05bd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:25.991853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:25.992812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:25.993690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:25.993690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:25.993690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:25.993690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:25.993690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:25.993690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:25.993690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:25.993690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:25.993690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:25.993690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:25.993690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:25.993690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:25.997041 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 61/127] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:38:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:25.998090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b43192-08c3-4802-9575-6466a0f05bd1 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:26.074210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:26.100888 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b1a791-968c-49ca-9317-f0ff9b040725 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:26.143967 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91b51b-5364-4d7c-9c94-bc91b2f8021c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:26.145133 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91b51b-5364-4d7c-9c94-bc91b2f8021c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:26.145559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91b51b-5364-4d7c-9c94-bc91b2f8021c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:26.145973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91b51b-5364-4d7c-9c94-bc91b2f8021c None admin] PATH_INFO: `/v3/roles/member` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:26.161802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91b51b-5364-4d7c-9c94-bc91b2f8021c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:26.167215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b43192-08c3-4802-9575-6466a0f05bd1 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-2c91b51b-5364-4d7c-9c94-bc91b2f8021c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:26.169219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:26.170632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:26.171494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:26.171494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:26.171494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:26.171494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:26.171494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:26.171494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:26.171494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:26.171494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:26.171494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:26.171494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:26.171494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:26.171494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:26.173918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 67/128] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:26.175041 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91b51b-5364-4d7c-9c94-bc91b2f8021c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:26.204393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:26.243907 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef877730-ebdd-4835-b16c-853289e8a6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:26.244604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef877730-ebdd-4835-b16c-853289e8a6ec None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:26.244892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef877730-ebdd-4835-b16c-853289e8a6ec None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:26.245177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef877730-ebdd-4835-b16c-853289e8a6ec None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea/users/89fc893d73ba473db202a90d053d6fd3/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:26.252051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef877730-ebdd-4835-b16c-853289e8a6ec None admin] RBAC: Authorizing `identity:create_grant(project_id=4681da4f1deb408293b2c948aec851ea, user_id=89fc893d73ba473db202a90d053d6fd3, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:26.255326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef877730-ebdd-4835-b16c-853289e8a6ec None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:26.281087 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 62/129] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:38:25 2022] PUT /identity/v3/projects/4681da4f1deb408293b2c948aec851ea/users/89fc893d73ba473db202a90d053d6fd3/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:38:26.284056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef877730-ebdd-4835-b16c-853289e8a6ec None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:26.354619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91b51b-5364-4d7c-9c94-bc91b2f8021c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:26.393844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f26dc-c1b9-4ecc-b928-36be8e11552c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:26.394563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f26dc-c1b9-4ecc-b928-36be8e11552c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:26.394858 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f26dc-c1b9-4ecc-b928-36be8e11552c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:26.395145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f26dc-c1b9-4ecc-b928-36be8e11552c None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:26.398246 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f26dc-c1b9-4ecc-b928-36be8e11552c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:26.398246 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f26dc-c1b9-4ecc-b928-36be8e11552c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:26.398246 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f26dc-c1b9-4ecc-b928-36be8e11552c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:26.407122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 68/130] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:26.413773 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f26dc-c1b9-4ecc-b928-36be8e11552c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:26.493692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:26.511348 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:26.527047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f26dc-c1b9-4ecc-b928-36be8e11552c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:26.530841 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef877730-ebdd-4835-b16c-853289e8a6ec None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:26.564225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c736063-df3c-4795-bf95-8a5bf50de400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:26.564951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c736063-df3c-4795-bf95-8a5bf50de400 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:26.565239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c736063-df3c-4795-bf95-8a5bf50de400 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:26.565579 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c736063-df3c-4795-bf95-8a5bf50de400 None admin] PATH_INFO: `/v3/users/7d770189a7bb4b81b8a42702f35fc84d` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:26.567119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c736063-df3c-4795-bf95-8a5bf50de400 None admin] RBAC: Authorizing `identity:get_user(user_id=7d770189a7bb4b81b8a42702f35fc84d)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:26.570344 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c736063-df3c-4795-bf95-8a5bf50de400 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:26.570344 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 69/131] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:26 2022] GET /identity/v3/users/7d770189a7bb4b81b8a42702f35fc84d => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:26.575916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c736063-df3c-4795-bf95-8a5bf50de400 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:26.601966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b1b39-a34e-46ce-95c2-a4f4d41fdb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:26.603268 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b1b39-a34e-46ce-95c2-a4f4d41fdb4f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:26.603771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b1b39-a34e-46ce-95c2-a4f4d41fdb4f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:26.604294 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b1b39-a34e-46ce-95c2-a4f4d41fdb4f None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:26.605309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b1b39-a34e-46ce-95c2-a4f4d41fdb4f None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:26.605985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b1b39-a34e-46ce-95c2-a4f4d41fdb4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:26.607574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b1b39-a34e-46ce-95c2-a4f4d41fdb4f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:26.626249 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 63/132] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:26.627839 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b1b39-a34e-46ce-95c2-a4f4d41fdb4f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:26.688765 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c736063-df3c-4795-bf95-8a5bf50de400 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:26.720998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:26.724374 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4be49-5236-4d88-914d-5221f694b246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:26.726143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4be49-5236-4d88-914d-5221f694b246 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:26.726143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4be49-5236-4d88-914d-5221f694b246 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:26.726143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4be49-5236-4d88-914d-5221f694b246 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:26.726666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4be49-5236-4d88-914d-5221f694b246 None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:26.728151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be4be49-5236-4d88-914d-5221f694b246 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:26.729998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 70/133] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:26 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:26.735498 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4be49-5236-4d88-914d-5221f694b246 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:26.790009 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b1b39-a34e-46ce-95c2-a4f4d41fdb4f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:26.837854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbfe54-7e06-41c9-87bd-db087b039103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:26.838740 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbfe54-7e06-41c9-87bd-db087b039103 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:26.839261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbfe54-7e06-41c9-87bd-db087b039103 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:26.839711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbfe54-7e06-41c9-87bd-db087b039103 None admin] PATH_INFO: `/v3/users/89fc893d73ba473db202a90d053d6fd3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:26.841416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbfe54-7e06-41c9-87bd-db087b039103 None admin] RBAC: Authorizing `identity:get_user(user_id=89fc893d73ba473db202a90d053d6fd3)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:26.842934 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4be49-5236-4d88-914d-5221f694b246 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:26.843620 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbfe54-7e06-41c9-87bd-db087b039103 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:26.845888 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 64/134] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:26 2022] GET /identity/v3/users/89fc893d73ba473db202a90d053d6fd3 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:26.857403 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbfe54-7e06-41c9-87bd-db087b039103 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:26.880006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76f3e0-e34c-48bd-9eaf-b6b2c284b16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:26.880734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76f3e0-e34c-48bd-9eaf-b6b2c284b16a None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:26.881026 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76f3e0-e34c-48bd-9eaf-b6b2c284b16a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:26.881315 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76f3e0-e34c-48bd-9eaf-b6b2c284b16a None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea/users/7d770189a7bb4b81b8a42702f35fc84d/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:26.883354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76f3e0-e34c-48bd-9eaf-b6b2c284b16a None admin] RBAC: Authorizing `identity:create_grant(project_id=4681da4f1deb408293b2c948aec851ea, user_id=7d770189a7bb4b81b8a42702f35fc84d, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:26.884917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db76f3e0-e34c-48bd-9eaf-b6b2c284b16a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:26.897427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:26.914867 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 71/135] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:38:26 2022] PUT /identity/v3/projects/4681da4f1deb408293b2c948aec851ea/users/7d770189a7bb4b81b8a42702f35fc84d/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:38:26.987816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbfe54-7e06-41c9-87bd-db087b039103 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:27.026051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a778ec9-766b-4c5a-8110-4e753c97b7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:27.026984 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a778ec9-766b-4c5a-8110-4e753c97b7cb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:27.027419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a778ec9-766b-4c5a-8110-4e753c97b7cb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:27.028527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a778ec9-766b-4c5a-8110-4e753c97b7cb None admin] PATH_INFO: `/v3/projects/950c54d4e4e44452809264618dea23df` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:27.029986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a778ec9-766b-4c5a-8110-4e753c97b7cb None admin] RBAC: Authorizing `identity:get_project(project_id=950c54d4e4e44452809264618dea23df)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:27.031615 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a778ec9-766b-4c5a-8110-4e753c97b7cb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:27.033525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 65/136] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:26 2022] GET /identity/v3/projects/950c54d4e4e44452809264618dea23df => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:27.039625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76f3e0-e34c-48bd-9eaf-b6b2c284b16a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:27.147561 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:27.199146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76f3e0-e34c-48bd-9eaf-b6b2c284b16a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:27.242373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcc07b-d71e-465d-8b64-fd365d479217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:27.243057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcc07b-d71e-465d-8b64-fd365d479217 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:27.243343 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcc07b-d71e-465d-8b64-fd365d479217 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:27.243623 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcc07b-d71e-465d-8b64-fd365d479217 None admin] PATH_INFO: `/v3/projects/950c54d4e4e44452809264618dea23df/users/89fc893d73ba473db202a90d053d6fd3/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:27.245603 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcc07b-d71e-465d-8b64-fd365d479217 None admin] RBAC: Authorizing `identity:create_grant(project_id=950c54d4e4e44452809264618dea23df, user_id=89fc893d73ba473db202a90d053d6fd3, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:27.247604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcc07b-d71e-465d-8b64-fd365d479217 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:27.265428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 72/137] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:38:27 2022] PUT /identity/v3/projects/950c54d4e4e44452809264618dea23df/users/89fc893d73ba473db202a90d053d6fd3/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:38:27.824259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d268c19-4bae-4c2d-aee0-3b70e42cd045 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:27.825926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d268c19-4bae-4c2d-aee0-3b70e42cd045 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:27.826412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d268c19-4bae-4c2d-aee0-3b70e42cd045 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:27.828264 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 66/138] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:27.842024 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44d659-cd03-4675-a4aa-19db3c0c331a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:27.842951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44d659-cd03-4675-a4aa-19db3c0c331a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:27.843446 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44d659-cd03-4675-a4aa-19db3c0c331a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:27.847255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d5e16-77c8-4aef-a4af-0da0cbb7fe39 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:27.848164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d5e16-77c8-4aef-a4af-0da0cbb7fe39 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:27.849238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d5e16-77c8-4aef-a4af-0da0cbb7fe39 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:27.853195 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 67/139] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:27.856779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacdf92-d0b9-49b4-ac84-d09ccf438df0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:27.857297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacdf92-d0b9-49b4-ac84-d09ccf438df0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:27.857873 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacdf92-d0b9-49b4-ac84-d09ccf438df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:27.893387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0e44d659-cd03-4675-a4aa-19db3c0c331a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:27.894944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0e44d659-cd03-4675-a4aa-19db3c0c331a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:27.901507 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-aaacdf92-d0b9-49b4-ac84-d09ccf438df0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:27.904377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-aaacdf92-d0b9-49b4-ac84-d09ccf438df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:28.067012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 73/140] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:28.078313 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 68/141] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:28.476185 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44d659-cd03-4675-a4aa-19db3c0c331a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:28.529262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:28.584652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44d659-cd03-4675-a4aa-19db3c0c331a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:28.609703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacdf92-d0b9-49b4-ac84-d09ccf438df0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:28.622685 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cb546-4c61-4bd2-b767-72fe7e73b004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:28.623334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cb546-4c61-4bd2-b767-72fe7e73b004 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:28.623617 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cb546-4c61-4bd2-b767-72fe7e73b004 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:28.623922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374cb546-4c61-4bd2-b767-72fe7e73b004 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:28.625039 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cb546-4c61-4bd2-b767-72fe7e73b004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:28.626557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374cb546-4c61-4bd2-b767-72fe7e73b004 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:28.628375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 74/142] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:28.633626 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cb546-4c61-4bd2-b767-72fe7e73b004 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:28.667474 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:28.740098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374cb546-4c61-4bd2-b767-72fe7e73b004 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:28.761605 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:28.783933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5de52a-987e-4dd3-ad96-078e3deae199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:28.784607 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5de52a-987e-4dd3-ad96-078e3deae199 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:28.784896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5de52a-987e-4dd3-ad96-078e3deae199 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:28.785172 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5de52a-987e-4dd3-ad96-078e3deae199 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:28.785845 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5de52a-987e-4dd3-ad96-078e3deae199 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:28.789839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5de52a-987e-4dd3-ad96-078e3deae199 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:28.809743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacdf92-d0b9-49b4-ac84-d09ccf438df0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:28.842254 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 75/143] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:38:28 2022] POST /identity/v3/users => generated 283 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:38:28.875840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451b009-a495-4022-8a33-de1ab19fe3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:28.876542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451b009-a495-4022-8a33-de1ab19fe3e8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:28.876828 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451b009-a495-4022-8a33-de1ab19fe3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:28.877103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451b009-a495-4022-8a33-de1ab19fe3e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:28.898095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451b009-a495-4022-8a33-de1ab19fe3e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-0451b009-a495-4022-8a33-de1ab19fe3e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:28.903758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:28.904779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:28.905584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:28.905584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:28.905584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:28.905584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:28.905584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:28.905584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:28.905584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:28.905584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:28.905584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:28.905584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:28.905584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:28.905584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:28.905584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 69/144] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:28.912598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5de52a-987e-4dd3-ad96-078e3deae199 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:29.037495 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5de52a-987e-4dd3-ad96-078e3deae199 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:29.082060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ff061-20e5-4062-87c1-7a1b59457b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:29.082683 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ff061-20e5-4062-87c1-7a1b59457b1b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:29.082969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ff061-20e5-4062-87c1-7a1b59457b1b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:29.083245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ff061-20e5-4062-87c1-7a1b59457b1b None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:29.086156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ff061-20e5-4062-87c1-7a1b59457b1b None admin] RBAC: Adding query filter params (name=reader) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:29.086156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ff061-20e5-4062-87c1-7a1b59457b1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:29.086156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66ff061-20e5-4062-87c1-7a1b59457b1b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:29.095304 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 76/145] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:29.101072 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451b009-a495-4022-8a33-de1ab19fe3e8 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:29.125485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cac56-5528-42ff-970c-f33b888dd7ae None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:29.125806 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cac56-5528-42ff-970c-f33b888dd7ae None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:29.126047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cac56-5528-42ff-970c-f33b888dd7ae None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:29.127708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 77/146] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:29.132976 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6d2f5-6fb8-43b8-b3ea-64509eee6369 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:29.133265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6d2f5-6fb8-43b8-b3ea-64509eee6369 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:29.133548 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6d2f5-6fb8-43b8-b3ea-64509eee6369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:29.161102 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-eab6d2f5-6fb8-43b8-b3ea-64509eee6369 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:29.162314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-eab6d2f5-6fb8-43b8-b3ea-64509eee6369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:29.221153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:29.263677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:29.294960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451b009-a495-4022-8a33-de1ab19fe3e8 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:29.308400 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 78/147] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:29.348998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3051e1-6278-4b9c-bb6a-ea95bad18a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:29.349637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3051e1-6278-4b9c-bb6a-ea95bad18a90 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:29.349923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3051e1-6278-4b9c-bb6a-ea95bad18a90 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:29.350204 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3051e1-6278-4b9c-bb6a-ea95bad18a90 None admin] PATH_INFO: `/v3/users/c2dcd642c2e24ebdacea43238f322e82` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:29.354050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3051e1-6278-4b9c-bb6a-ea95bad18a90 None admin] RBAC: Authorizing `identity:get_user(user_id=c2dcd642c2e24ebdacea43238f322e82)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:29.355534 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3051e1-6278-4b9c-bb6a-ea95bad18a90 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:29.357230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 70/148] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:29 2022] GET /identity/v3/users/c2dcd642c2e24ebdacea43238f322e82 => generated 289 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:29.362698 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6d2f5-6fb8-43b8-b3ea-64509eee6369 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:29.437798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf248c0f-53d4-45a0-ac70-2c064b3cc263 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:29.438121 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf248c0f-53d4-45a0-ac70-2c064b3cc263 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:29.438373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf248c0f-53d4-45a0-ac70-2c064b3cc263 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:29.439865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 71/149] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:38:29.444940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caba92-9385-48a3-9f34-6bca47268e9d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:29.445234 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caba92-9385-48a3-9f34-6bca47268e9d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:29.445524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88caba92-9385-48a3-9f34-6bca47268e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:29.470852 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-88caba92-9385-48a3-9f34-6bca47268e9d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:29.472087 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-88caba92-9385-48a3-9f34-6bca47268e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:29.522074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6d2f5-6fb8-43b8-b3ea-64509eee6369 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:29.567734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ee5b6-3219-4a59-a17a-f4c3c080751d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:29.568382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ee5b6-3219-4a59-a17a-f4c3c080751d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:29.568667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ee5b6-3219-4a59-a17a-f4c3c080751d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:29.568941 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ee5b6-3219-4a59-a17a-f4c3c080751d None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:29.570148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ee5b6-3219-4a59-a17a-f4c3c080751d None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:29.571610 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ee5b6-3219-4a59-a17a-f4c3c080751d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:29.573295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 79/150] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:29 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:29.581151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ee5b6-3219-4a59-a17a-f4c3c080751d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:29.633931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 72/151] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:29.652190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:29.717021 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88caba92-9385-48a3-9f34-6bca47268e9d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:29.730596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ee5b6-3219-4a59-a17a-f4c3c080751d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:29.772143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809449ee-d07f-458c-9993-3bd38fbc0f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:29.772821 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809449ee-d07f-458c-9993-3bd38fbc0f9a None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:29.773105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809449ee-d07f-458c-9993-3bd38fbc0f9a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:29.773380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809449ee-d07f-458c-9993-3bd38fbc0f9a None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea/users/c2dcd642c2e24ebdacea43238f322e82/roles/2235d4074114441ea4661c3fb936dcc3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:29.776223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809449ee-d07f-458c-9993-3bd38fbc0f9a None admin] RBAC: Authorizing `identity:create_grant(project_id=4681da4f1deb408293b2c948aec851ea, user_id=c2dcd642c2e24ebdacea43238f322e82, role_id=2235d4074114441ea4661c3fb936dcc3)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:29.777839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809449ee-d07f-458c-9993-3bd38fbc0f9a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:29.808479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 80/152] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:38:29 2022] PUT /identity/v3/projects/4681da4f1deb408293b2c948aec851ea/users/c2dcd642c2e24ebdacea43238f322e82/roles/2235d4074114441ea4661c3fb936dcc3 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:38:29.863346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:29.885342 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88caba92-9385-48a3-9f34-6bca47268e9d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:29.926076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e9104-6682-423f-a5da-772ac267ad63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:29.926706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e9104-6682-423f-a5da-772ac267ad63 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:29.926989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e9104-6682-423f-a5da-772ac267ad63 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:29.927267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e9104-6682-423f-a5da-772ac267ad63 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:29.939927 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e9104-6682-423f-a5da-772ac267ad63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-026e9104-6682-423f-a5da-772ac267ad63 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:29.944979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:29.947565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:29.950030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:29.950030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:29.950030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:29.950030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:29.950030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:29.950030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:29.950030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:29.950030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:29.950030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:29.950030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:29.950030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:38:29.950030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:29.950030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 73/153] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:38:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:38:29.953694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809449ee-d07f-458c-9993-3bd38fbc0f9a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:30.016694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e9104-6682-423f-a5da-772ac267ad63 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:30.088928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809449ee-d07f-458c-9993-3bd38fbc0f9a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:30.092140 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:30.145644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff994f-c3e4-4c70-9147-ee39151bda65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:30.146373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff994f-c3e4-4c70-9147-ee39151bda65 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:30.146657 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff994f-c3e4-4c70-9147-ee39151bda65 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:30.146926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff994f-c3e4-4c70-9147-ee39151bda65 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:30.147606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff994f-c3e4-4c70-9147-ee39151bda65 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:30.148227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff994f-c3e4-4c70-9147-ee39151bda65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:30.149490 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff994f-c3e4-4c70-9147-ee39151bda65 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:30.165315 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 81/154] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:38:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:30.171480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff994f-c3e4-4c70-9147-ee39151bda65 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:30.209135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e9104-6682-423f-a5da-772ac267ad63 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:30.279435 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc8e85-52c5-4a2d-b178-d1f848e98e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:30.280109 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc8e85-52c5-4a2d-b178-d1f848e98e8b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:30.280387 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc8e85-52c5-4a2d-b178-d1f848e98e8b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:30.280690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc8e85-52c5-4a2d-b178-d1f848e98e8b None admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:30.296093 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc8e85-52c5-4a2d-b178-d1f848e98e8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-13bc8e85-52c5-4a2d-b178-d1f848e98e8b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:30.301135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:30.303156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:30.305287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:30.305287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:30.305287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:30.305287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:30.305287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:30.305287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:30.305287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:30.305287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:30.305287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:30.305287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:30.305287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:30.305287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:30.311943 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 74/155] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:38:30.311943 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc8e85-52c5-4a2d-b178-d1f848e98e8b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:30.315327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:30.317948 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff994f-c3e4-4c70-9147-ee39151bda65 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:30.374790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb5fd7-5336-4cac-9dcc-1f8153cfdce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:30.375440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb5fd7-5336-4cac-9dcc-1f8153cfdce8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:30.375721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb5fd7-5336-4cac-9dcc-1f8153cfdce8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:30.376029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb5fd7-5336-4cac-9dcc-1f8153cfdce8 None admin] PATH_INFO: `/v3/users/89fc893d73ba473db202a90d053d6fd3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:30.380650 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb5fd7-5336-4cac-9dcc-1f8153cfdce8 None admin] RBAC: Authorizing `identity:get_user(user_id=89fc893d73ba473db202a90d053d6fd3)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:30.382125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb5fd7-5336-4cac-9dcc-1f8153cfdce8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:30.384003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 82/156] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:30 2022] GET /identity/v3/users/89fc893d73ba473db202a90d053d6fd3 => generated 275 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:30.391064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb5fd7-5336-4cac-9dcc-1f8153cfdce8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:30.430869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:30.436333 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc8e85-52c5-4a2d-b178-d1f848e98e8b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:30.477559 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383059f-ebcb-499f-8dbb-73ea6ace328d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:30.478439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383059f-ebcb-499f-8dbb-73ea6ace328d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:30.478867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383059f-ebcb-499f-8dbb-73ea6ace328d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:30.479274 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383059f-ebcb-499f-8dbb-73ea6ace328d None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:30.480108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383059f-ebcb-499f-8dbb-73ea6ace328d None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:30.481288 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383059f-ebcb-499f-8dbb-73ea6ace328d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:30.483155 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383059f-ebcb-499f-8dbb-73ea6ace328d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:30.496081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 75/157] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:30.498244 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c971d9-2574-4684-97b8-06398d82a5c9 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:30.498829 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c971d9-2574-4684-97b8-06398d82a5c9 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:30.499277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c971d9-2574-4684-97b8-06398d82a5c9 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:30.502059 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 76/158] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:30.505174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c971d9-2574-4684-97b8-06398d82a5c9 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:30.561635 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb5fd7-5336-4cac-9dcc-1f8153cfdce8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:30.583651 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:30.619710 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f93a2-35b2-49bd-b527-4167f20212ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:30.620777 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f93a2-35b2-49bd-b527-4167f20212ec None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:30.621065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f93a2-35b2-49bd-b527-4167f20212ec None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:30.621389 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f93a2-35b2-49bd-b527-4167f20212ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:30.634761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c971d9-2574-4684-97b8-06398d82a5c9 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:30.688925 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9e0dc-4d64-4998-b7e5-339ed3b69367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:30.690421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9e0dc-4d64-4998-b7e5-339ed3b69367 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:30.690849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9e0dc-4d64-4998-b7e5-339ed3b69367 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:30.691260 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9e0dc-4d64-4998-b7e5-339ed3b69367 None admin] PATH_INFO: `/v3/users/89fc893d73ba473db202a90d053d6fd3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:30.692574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9e0dc-4d64-4998-b7e5-339ed3b69367 None admin] RBAC: Authorizing `identity:get_user(user_id=89fc893d73ba473db202a90d053d6fd3)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:30.694720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9e0dc-4d64-4998-b7e5-339ed3b69367 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:30.699969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 77/159] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:30 2022] GET /identity/v3/users/89fc893d73ba473db202a90d053d6fd3 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:30.701524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08371839-eec8-40c2-88a0-b806327c67ad None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:30.701807 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08371839-eec8-40c2-88a0-b806327c67ad None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:30.703296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08371839-eec8-40c2-88a0-b806327c67ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:30.721940 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f93a2-35b2-49bd-b527-4167f20212ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:30.724689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214f93a2-35b2-49bd-b527-4167f20212ec None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:30.728416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-08371839-eec8-40c2-88a0-b806327c67ad None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:30.729235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-08371839-eec8-40c2-88a0-b806327c67ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:30.755458 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:30.810085 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 78/160] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:30.811120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08371839-eec8-40c2-88a0-b806327c67ad None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:30.840683 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 83/161] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 14:38:30.855604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f93a2-35b2-49bd-b527-4167f20212ec None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:30.907407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:30.969994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08371839-eec8-40c2-88a0-b806327c67ad None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:30.972046 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f93a2-35b2-49bd-b527-4167f20212ec None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:30.999920 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:31.021621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285837a7-e3d1-4d31-9136-7e5d339ef6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:31.022506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37a2e3-0bd6-426f-9ff6-2fb1de6d0f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:31.022685 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285837a7-e3d1-4d31-9136-7e5d339ef6ba None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:31.023099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285837a7-e3d1-4d31-9136-7e5d339ef6ba None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:31.023441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37a2e3-0bd6-426f-9ff6-2fb1de6d0f14 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:31.023676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285837a7-e3d1-4d31-9136-7e5d339ef6ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:31.023974 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37a2e3-0bd6-426f-9ff6-2fb1de6d0f14 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:31.024342 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37a2e3-0bd6-426f-9ff6-2fb1de6d0f14 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:31.029476 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37a2e3-0bd6-426f-9ff6-2fb1de6d0f14 None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:31.030668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37a2e3-0bd6-426f-9ff6-2fb1de6d0f14 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:31.032588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 84/162] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:30 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:31.048963 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37a2e3-0bd6-426f-9ff6-2fb1de6d0f14 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:31.107564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285837a7-e3d1-4d31-9136-7e5d339ef6ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:31.109367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285837a7-e3d1-4d31-9136-7e5d339ef6ba None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:31.198667 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:31.244773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 79/163] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:38:31.249231 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37a2e3-0bd6-426f-9ff6-2fb1de6d0f14 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:31.251332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285837a7-e3d1-4d31-9136-7e5d339ef6ba None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:31.347317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dea376-94dd-49f9-8406-f12f12b7e026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:31.347966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dea376-94dd-49f9-8406-f12f12b7e026 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:31.348246 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dea376-94dd-49f9-8406-f12f12b7e026 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:31.348518 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dea376-94dd-49f9-8406-f12f12b7e026 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:31.349244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dea376-94dd-49f9-8406-f12f12b7e026 None admin] RBAC: Adding query filter params (role.id=fe3bbad209634f16b1fcb66d6d49f665, scope.project.id=4681da4f1deb408293b2c948aec851ea, user.id=89fc893d73ba473db202a90d053d6fd3) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:31.349568 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dea376-94dd-49f9-8406-f12f12b7e026 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:31.351011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dea376-94dd-49f9-8406-f12f12b7e026 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:31.362935 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 85/164] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:31 2022] GET /identity/v3/role_assignments?user.id=89fc893d73ba473db202a90d053d6fd3&scope.project.id=4681da4f1deb408293b2c948aec851ea&role.id=fe3bbad209634f16b1fcb66d6d49f665 => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:31.367354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ab32e-23bc-4c63-9768-3e01983d74dc None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:31.367354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ab32e-23bc-4c63-9768-3e01983d74dc None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:31.367354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ab32e-23bc-4c63-9768-3e01983d74dc None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:31.367354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 86/165] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:31.372266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e27d1-bb3b-4c9c-a4fd-df2d78218bd3 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:31.372554 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e27d1-bb3b-4c9c-a4fd-df2d78218bd3 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:31.372828 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e27d1-bb3b-4c9c-a4fd-df2d78218bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:31.404284 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-420e27d1-bb3b-4c9c-a4fd-df2d78218bd3 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:31.405579 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-420e27d1-bb3b-4c9c-a4fd-df2d78218bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:31.408107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285837a7-e3d1-4d31-9136-7e5d339ef6ba None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:31.476153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654db7e-238b-4af5-abf6-c0bfbe03c74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:31.477315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654db7e-238b-4af5-abf6-c0bfbe03c74a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:31.477794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654db7e-238b-4af5-abf6-c0bfbe03c74a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:31.478259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654db7e-238b-4af5-abf6-c0bfbe03c74a None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:31.480479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654db7e-238b-4af5-abf6-c0bfbe03c74a None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:31.482760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654db7e-238b-4af5-abf6-c0bfbe03c74a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:31.485829 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 80/166] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:31 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:31.491774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654db7e-238b-4af5-abf6-c0bfbe03c74a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:31.549194 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:31.552573 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:31.586233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 87/167] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:31.587113 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e27d1-bb3b-4c9c-a4fd-df2d78218bd3 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:31.667173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654db7e-238b-4af5-abf6-c0bfbe03c74a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:31.720566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d85d1-5a96-4686-a548-7a0be7c6e827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:31.721193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d85d1-5a96-4686-a548-7a0be7c6e827 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:31.721473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d85d1-5a96-4686-a548-7a0be7c6e827 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:31.721746 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d85d1-5a96-4686-a548-7a0be7c6e827 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:31.722653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d85d1-5a96-4686-a548-7a0be7c6e827 None admin] RBAC: Adding query filter params (role.id=0a6592b9a0b64df6938aba0a010c2256, scope.project.id=4681da4f1deb408293b2c948aec851ea, user.id=89fc893d73ba473db202a90d053d6fd3) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:31.722966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d85d1-5a96-4686-a548-7a0be7c6e827 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:31.728684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d85d1-5a96-4686-a548-7a0be7c6e827 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:31.736896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 81/168] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:31 2022] GET /identity/v3/role_assignments?user.id=89fc893d73ba473db202a90d053d6fd3&scope.project.id=4681da4f1deb408293b2c948aec851ea&role.id=0a6592b9a0b64df6938aba0a010c2256 => generated 608 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:31.818441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e27d1-bb3b-4c9c-a4fd-df2d78218bd3 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:31.863003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb1bde-c555-45d2-a397-a3a789e11bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:31.863706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb1bde-c555-45d2-a397-a3a789e11bf0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:31.864085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb1bde-c555-45d2-a397-a3a789e11bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:31.864411 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb1bde-c555-45d2-a397-a3a789e11bf0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:31.877126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb1bde-c555-45d2-a397-a3a789e11bf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-26fb1bde-c555-45d2-a397-a3a789e11bf0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:31.882380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:31.883317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:31.884850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:31.884850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:31.884850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:31.884850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:31.884850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:31.884850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:31.884850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:31.884850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:31.884850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:31.884850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:31.884850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:31.884850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:31.884850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 88/169] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:38:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:31.889674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d85d1-5a96-4686-a548-7a0be7c6e827 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:32.005589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:32.014200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d85d1-5a96-4686-a548-7a0be7c6e827 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:32.053032 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5176d-da26-4ccb-8c81-88fbac980349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:32.053659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5176d-da26-4ccb-8c81-88fbac980349 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:32.053940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5176d-da26-4ccb-8c81-88fbac980349 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:32.054212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5176d-da26-4ccb-8c81-88fbac980349 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:32.054918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5176d-da26-4ccb-8c81-88fbac980349 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:32.055220 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5176d-da26-4ccb-8c81-88fbac980349 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:32.056757 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5176d-da26-4ccb-8c81-88fbac980349 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:32.066122 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 82/170] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:32.073073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb1bde-c555-45d2-a397-a3a789e11bf0 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:32.147472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:32.221313 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5176d-da26-4ccb-8c81-88fbac980349 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:32.222664 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb1bde-c555-45d2-a397-a3a789e11bf0 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:32.277501 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f474a5-04e3-4037-9e35-dec0e4223527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:32.278774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f474a5-04e3-4037-9e35-dec0e4223527 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:32.279057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f474a5-04e3-4037-9e35-dec0e4223527 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:32.279328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f474a5-04e3-4037-9e35-dec0e4223527 None admin] PATH_INFO: `/v3/users/7d770189a7bb4b81b8a42702f35fc84d` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:32.280611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f474a5-04e3-4037-9e35-dec0e4223527 None admin] RBAC: Authorizing `identity:get_user(user_id=7d770189a7bb4b81b8a42702f35fc84d)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:32.283007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f474a5-04e3-4037-9e35-dec0e4223527 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:32.284872 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 89/171] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:32 2022] GET /identity/v3/users/7d770189a7bb4b81b8a42702f35fc84d => generated 247 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:32.293138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f474a5-04e3-4037-9e35-dec0e4223527 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:32.417884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5176d-da26-4ccb-8c81-88fbac980349 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:32.495815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28104e25-8252-4d63-bd4e-eac9f3bd13ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:32.497000 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28104e25-8252-4d63-bd4e-eac9f3bd13ea None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:32.497281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28104e25-8252-4d63-bd4e-eac9f3bd13ea None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:32.497552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28104e25-8252-4d63-bd4e-eac9f3bd13ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:32.514945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28104e25-8252-4d63-bd4e-eac9f3bd13ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-28104e25-8252-4d63-bd4e-eac9f3bd13ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:32.520243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:32.521227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:32.522659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:32.522659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:32.522659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:32.522659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:32.522659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:32.522659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:32.522659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:32.522659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:32.522659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:32.522659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:32.522659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:32.522659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:32.522659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 83/172] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:32.526163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f474a5-04e3-4037-9e35-dec0e4223527 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:32.529043 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28104e25-8252-4d63-bd4e-eac9f3bd13ea None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:32.556351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:32.566866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:32.601565 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac5d23-0449-4b4a-9ca2-a1ac12d9826e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:32.602189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac5d23-0449-4b4a-9ca2-a1ac12d9826e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:32.602509 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac5d23-0449-4b4a-9ca2-a1ac12d9826e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:32.603282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac5d23-0449-4b4a-9ca2-a1ac12d9826e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:32.750251 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac5d23-0449-4b4a-9ca2-a1ac12d9826e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:32.751629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac5d23-0449-4b4a-9ca2-a1ac12d9826e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:32.753908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28104e25-8252-4d63-bd4e-eac9f3bd13ea None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:32.817996 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7be80b-bc2b-49f9-a1ff-3361dc01b7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:32.818614 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7be80b-bc2b-49f9-a1ff-3361dc01b7e2 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:32.818893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7be80b-bc2b-49f9-a1ff-3361dc01b7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:32.819168 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7be80b-bc2b-49f9-a1ff-3361dc01b7e2 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:32.819850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7be80b-bc2b-49f9-a1ff-3361dc01b7e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:32.820161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7be80b-bc2b-49f9-a1ff-3361dc01b7e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:32.821576 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7be80b-bc2b-49f9-a1ff-3361dc01b7e2 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:32.837106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 84/173] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:32.842848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7be80b-bc2b-49f9-a1ff-3361dc01b7e2 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:32.912564 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 90/174] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 14:38:32.919102 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac5d23-0449-4b4a-9ca2-a1ac12d9826e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:32.960193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:32.964151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:32.999628 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7be80b-bc2b-49f9-a1ff-3361dc01b7e2 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:33.064798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018b0af-0b30-40bd-b346-ce0033eaf536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:33.065428 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018b0af-0b30-40bd-b346-ce0033eaf536 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:33.065659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018b0af-0b30-40bd-b346-ce0033eaf536 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:33.065914 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6018b0af-0b30-40bd-b346-ce0033eaf536 None admin] PATH_INFO: `/v3/users/89fc893d73ba473db202a90d053d6fd3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:33.067264 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018b0af-0b30-40bd-b346-ce0033eaf536 None admin] RBAC: Authorizing `identity:get_user(user_id=89fc893d73ba473db202a90d053d6fd3)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:33.068883 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6018b0af-0b30-40bd-b346-ce0033eaf536 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:33.070758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 85/175] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:32 2022] GET /identity/v3/users/89fc893d73ba473db202a90d053d6fd3 => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:33.079848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018b0af-0b30-40bd-b346-ce0033eaf536 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:33.145541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac5d23-0449-4b4a-9ca2-a1ac12d9826e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:33.191666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b67f2-4bbc-41e9-a2e2-cc599fea25c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:33.192348 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b67f2-4bbc-41e9-a2e2-cc599fea25c2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:33.192632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b67f2-4bbc-41e9-a2e2-cc599fea25c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:33.192909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b67f2-4bbc-41e9-a2e2-cc599fea25c2 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:33.194046 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b67f2-4bbc-41e9-a2e2-cc599fea25c2 None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:33.195621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b67f2-4bbc-41e9-a2e2-cc599fea25c2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:33.197399 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 91/176] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:32 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:33.204057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b67f2-4bbc-41e9-a2e2-cc599fea25c2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:33.349264 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6018b0af-0b30-40bd-b346-ce0033eaf536 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:33.364202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:33.407555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b67f2-4bbc-41e9-a2e2-cc599fea25c2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:33.425452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eeda6e-42a6-4dcd-a7db-e1ebf37dfcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:33.426067 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeda6e-42a6-4dcd-a7db-e1ebf37dfcbe None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:33.426345 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeda6e-42a6-4dcd-a7db-e1ebf37dfcbe None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:33.426618 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeda6e-42a6-4dcd-a7db-e1ebf37dfcbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:33.457112 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:33.483352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda8caa-6275-4e96-811b-ec460b7fc6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:33.483997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda8caa-6275-4e96-811b-ec460b7fc6cb None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:33.484279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda8caa-6275-4e96-811b-ec460b7fc6cb None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:33.484553 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda8caa-6275-4e96-811b-ec460b7fc6cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:33.485289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda8caa-6275-4e96-811b-ec460b7fc6cb None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=4681da4f1deb408293b2c948aec851ea, user.id=7d770189a7bb4b81b8a42702f35fc84d) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:33.485595 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda8caa-6275-4e96-811b-ec460b7fc6cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:33.487031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda8caa-6275-4e96-811b-ec460b7fc6cb None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:33.500904 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 92/177] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:33 2022] GET /identity/v3/role_assignments?user.id=7d770189a7bb4b81b8a42702f35fc84d&scope.project.id=4681da4f1deb408293b2c948aec851ea&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:33.560447 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eeda6e-42a6-4dcd-a7db-e1ebf37dfcbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:33.561931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eeda6e-42a6-4dcd-a7db-e1ebf37dfcbe None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:33.689043 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 86/178] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 14:38:33.697284 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda8caa-6275-4e96-811b-ec460b7fc6cb None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:33.829143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda8caa-6275-4e96-811b-ec460b7fc6cb None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:33.879778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd2cfcb-8cf2-4fec-ae1c-509724249d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:33.880458 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2cfcb-8cf2-4fec-ae1c-509724249d8f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:33.880737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2cfcb-8cf2-4fec-ae1c-509724249d8f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:33.881012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2cfcb-8cf2-4fec-ae1c-509724249d8f None admin] PATH_INFO: `/v3/projects/950c54d4e4e44452809264618dea23df` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:33.882350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2cfcb-8cf2-4fec-ae1c-509724249d8f None admin] RBAC: Authorizing `identity:get_project(project_id=950c54d4e4e44452809264618dea23df)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:33.883947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd2cfcb-8cf2-4fec-ae1c-509724249d8f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:33.885644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 93/179] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:33 2022] GET /identity/v3/projects/950c54d4e4e44452809264618dea23df => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:33.891781 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eeda6e-42a6-4dcd-a7db-e1ebf37dfcbe None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:33.939452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e740d-03ee-43c1-adec-daba1fe83c7b None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:33.939748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e740d-03ee-43c1-adec-daba1fe83c7b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:33.939950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e740d-03ee-43c1-adec-daba1fe83c7b None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:33.941315 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 94/180] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:38:33.947938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aec45-8240-4357-89eb-ee03de0704a8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:33.948246 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aec45-8240-4357-89eb-ee03de0704a8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:33.948562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685aec45-8240-4357-89eb-ee03de0704a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:33.976236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:33.978280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-685aec45-8240-4357-89eb-ee03de0704a8 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:33.981589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-685aec45-8240-4357-89eb-ee03de0704a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:33.998535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:34.063539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eeda6e-42a6-4dcd-a7db-e1ebf37dfcbe None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:34.078342 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 95/181] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:34.080119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74436d3b-0145-4ff5-8b87-75c780a3cc78 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:34.080568 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74436d3b-0145-4ff5-8b87-75c780a3cc78 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:34.080989 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74436d3b-0145-4ff5-8b87-75c780a3cc78 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:34.082561 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 96/182] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:34.088099 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868eb582-d95d-42f0-bea5-f1cb3d3d4c31 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:34.088718 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868eb582-d95d-42f0-bea5-f1cb3d3d4c31 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:34.089148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868eb582-d95d-42f0-bea5-f1cb3d3d4c31 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:34.091904 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 97/183] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:34.093343 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaeae9b-0d32-434c-80e2-b42a064df4a4 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:34.093621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaeae9b-0d32-434c-80e2-b42a064df4a4 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:34.093898 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaeae9b-0d32-434c-80e2-b42a064df4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:34.113597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49aab5d-4e9f-4a52-97c0-95ef7c6fe92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:34.114925 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49aab5d-4e9f-4a52-97c0-95ef7c6fe92b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:34.115415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49aab5d-4e9f-4a52-97c0-95ef7c6fe92b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:34.115937 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49aab5d-4e9f-4a52-97c0-95ef7c6fe92b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:34.116898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49aab5d-4e9f-4a52-97c0-95ef7c6fe92b None admin] RBAC: Adding query filter params (role.id=0a6592b9a0b64df6938aba0a010c2256, scope.project.id=950c54d4e4e44452809264618dea23df, user.id=89fc893d73ba473db202a90d053d6fd3) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:34.117019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ccaeae9b-0d32-434c-80e2-b42a064df4a4 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:34.117533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49aab5d-4e9f-4a52-97c0-95ef7c6fe92b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:34.118106 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ccaeae9b-0d32-434c-80e2-b42a064df4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:34.119421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49aab5d-4e9f-4a52-97c0-95ef7c6fe92b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:34.130128 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 87/184] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:33 2022] GET /identity/v3/role_assignments?user.id=89fc893d73ba473db202a90d053d6fd3&scope.project.id=950c54d4e4e44452809264618dea23df&role.id=0a6592b9a0b64df6938aba0a010c2256 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:34.132300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f1317-6c75-423a-83ac-04c665456e0e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:34.132821 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f1317-6c75-423a-83ac-04c665456e0e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:34.133296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f1317-6c75-423a-83ac-04c665456e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:34.162907 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-732f1317-6c75-423a-83ac-04c665456e0e None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:34.164629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-732f1317-6c75-423a-83ac-04c665456e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:34.290825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 88/185] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:34.296719 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88a0e7-3434-468d-b3b5-81457a841f0c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:34.297032 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88a0e7-3434-468d-b3b5-81457a841f0c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:34.297306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff88a0e7-3434-468d-b3b5-81457a841f0c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:34.298908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 89/186] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:34.306716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64436652-0da4-46b4-991c-5e26e2c29b2e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:34.307201 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64436652-0da4-46b4-991c-5e26e2c29b2e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:34.307694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64436652-0da4-46b4-991c-5e26e2c29b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:34.345955 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 98/187] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:34.364960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-64436652-0da4-46b4-991c-5e26e2c29b2e None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:34.366619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-64436652-0da4-46b4-991c-5e26e2c29b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:34.407509 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaeae9b-0d32-434c-80e2-b42a064df4a4 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:34.444178 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:34.499006 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 90/188] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:34.540223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:34.605133 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaeae9b-0d32-434c-80e2-b42a064df4a4 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:34.661687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525aef3-322c-4dc1-892d-6dc50646d999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:34.662335 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525aef3-322c-4dc1-892d-6dc50646d999 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:34.662614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525aef3-322c-4dc1-892d-6dc50646d999 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:34.662886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525aef3-322c-4dc1-892d-6dc50646d999 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:34.678445 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525aef3-322c-4dc1-892d-6dc50646d999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:34.679572 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64436652-0da4-46b4-991c-5e26e2c29b2e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-1525aef3-322c-4dc1-892d-6dc50646d999 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:34.683298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:34.685096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:34.687302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:34.687302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:34.687302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:34.687302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:34.687302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:34.687302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:34.687302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:34.687302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:34.687302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:34.687302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:34.687302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:34.687302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:34.687302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 99/189] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:34.692379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525aef3-322c-4dc1-892d-6dc50646d999 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:34.800074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64436652-0da4-46b4-991c-5e26e2c29b2e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:34.806092 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525aef3-322c-4dc1-892d-6dc50646d999 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:34.835090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b471ec3-3738-4ad8-b2a7-d92c8d2f5d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:34.835737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b471ec3-3738-4ad8-b2a7-d92c8d2f5d18 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:34.836051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b471ec3-3738-4ad8-b2a7-d92c8d2f5d18 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:34.836328 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b471ec3-3738-4ad8-b2a7-d92c8d2f5d18 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:34.841300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1b110-3fd3-47ee-9c39-046b5435e05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:34.841911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1b110-3fd3-47ee-9c39-046b5435e05c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:34.842189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1b110-3fd3-47ee-9c39-046b5435e05c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:34.842461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e1b110-3fd3-47ee-9c39-046b5435e05c None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:34.843121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1b110-3fd3-47ee-9c39-046b5435e05c None admin] RBAC: Adding query filter params (name=reader) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:34.843420 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1b110-3fd3-47ee-9c39-046b5435e05c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:34.844840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e1b110-3fd3-47ee-9c39-046b5435e05c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:34.846260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:34.847358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b471ec3-3738-4ad8-b2a7-d92c8d2f5d18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-4b471ec3-3738-4ad8-b2a7-d92c8d2f5d18 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:34.852257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:34.856349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:34.857281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:34.857281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:34.857281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:34.857281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:34.857281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:34.857281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:34.857281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:34.857281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:34.857281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:34.857281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:34.857281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:38:34.857281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:34.857281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 91/190] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:38:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:34.857971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 100/191] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:34.858042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b471ec3-3738-4ad8-b2a7-d92c8d2f5d18 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:34.858107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1b110-3fd3-47ee-9c39-046b5435e05c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:34.889178 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:34.957735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b471ec3-3738-4ad8-b2a7-d92c8d2f5d18 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:34.958244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e1b110-3fd3-47ee-9c39-046b5435e05c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:34.992597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d527c-700b-43b8-b2c9-7c07282076e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:34.992815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1957581-b452-4369-bc2b-cfe196adc788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:34.993208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d527c-700b-43b8-b2c9-7c07282076e5 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:34.993383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1957581-b452-4369-bc2b-cfe196adc788 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:34.993478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d527c-700b-43b8-b2c9-7c07282076e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:34.993662 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1957581-b452-4369-bc2b-cfe196adc788 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:34.993752 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d527c-700b-43b8-b2c9-7c07282076e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:34.993932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1957581-b452-4369-bc2b-cfe196adc788 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:34.994817 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d527c-700b-43b8-b2c9-7c07282076e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:34.996292 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d527c-700b-43b8-b2c9-7c07282076e5 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:34.998384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 101/192] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:35.000878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d527c-700b-43b8-b2c9-7c07282076e5 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:35.004815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1957581-b452-4369-bc2b-cfe196adc788 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-a1957581-b452-4369-bc2b-cfe196adc788 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:35.009683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:35.010782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:35.011616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:35.011616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:35.011616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:35.011616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:35.011616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:35.011616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:35.011616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:35.011616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:35.011616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:35.011616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:35.011616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:35.011616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:35.011616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 92/193] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:38:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:35.011616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1957581-b452-4369-bc2b-cfe196adc788 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:35.067728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:35.107437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:35.111566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d527c-700b-43b8-b2c9-7c07282076e5 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:35.117960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1957581-b452-4369-bc2b-cfe196adc788 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:35.145641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d8056-6371-4933-8109-b8bd97126bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:35.146248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d8056-6371-4933-8109-b8bd97126bb4 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:35.146531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d8056-6371-4933-8109-b8bd97126bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:35.146807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d8056-6371-4933-8109-b8bd97126bb4 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:35.147468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d8056-6371-4933-8109-b8bd97126bb4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:35.147770 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d8056-6371-4933-8109-b8bd97126bb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:35.149253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d8056-6371-4933-8109-b8bd97126bb4 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:35.152130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11ea03-9600-42e4-9391-795e059b181c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:35.152784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11ea03-9600-42e4-9391-795e059b181c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:35.153060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11ea03-9600-42e4-9391-795e059b181c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:35.153331 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11ea03-9600-42e4-9391-795e059b181c None admin] PATH_INFO: `/v3/users/c2dcd642c2e24ebdacea43238f322e82` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:35.154439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11ea03-9600-42e4-9391-795e059b181c None admin] RBAC: Authorizing `identity:get_user(user_id=c2dcd642c2e24ebdacea43238f322e82)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:35.155918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11ea03-9600-42e4-9391-795e059b181c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:35.157075 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 102/194] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:38:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:35.157921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d8056-6371-4933-8109-b8bd97126bb4 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:35.160280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 93/195] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:35 2022] GET /identity/v3/users/c2dcd642c2e24ebdacea43238f322e82 => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:35.161251 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11ea03-9600-42e4-9391-795e059b181c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:35.260638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11ea03-9600-42e4-9391-795e059b181c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:35.260844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d8056-6371-4933-8109-b8bd97126bb4 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:35.285251 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:35.295028 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc4021-ad08-43da-b377-e5944da9cc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:35.295199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ae2a7-1866-49df-b0b3-e08b54096e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:35.295913 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc4021-ad08-43da-b377-e5944da9cc23 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:35.296019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae2a7-1866-49df-b0b3-e08b54096e6b None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:35.296192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc4021-ad08-43da-b377-e5944da9cc23 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:35.296281 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae2a7-1866-49df-b0b3-e08b54096e6b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:35.296461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc4021-ad08-43da-b377-e5944da9cc23 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:35.296551 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ae2a7-1866-49df-b0b3-e08b54096e6b None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:35.297113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc4021-ad08-43da-b377-e5944da9cc23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:35.297210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ae2a7-1866-49df-b0b3-e08b54096e6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:35.297409 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc4021-ad08-43da-b377-e5944da9cc23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:35.298592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ae2a7-1866-49df-b0b3-e08b54096e6b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:35.298772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc4021-ad08-43da-b377-e5944da9cc23 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:35.306754 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 94/196] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:35.307546 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc4021-ad08-43da-b377-e5944da9cc23 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:35.321498 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 103/197] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:38:35 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:38:35.328968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ae2a7-1866-49df-b0b3-e08b54096e6b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:35.334115 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:35.425262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc4021-ad08-43da-b377-e5944da9cc23 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:35.445555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ae2a7-1866-49df-b0b3-e08b54096e6b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:35.468737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f810e16-cc9e-4610-ae42-483c194c1be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:35.469385 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f810e16-cc9e-4610-ae42-483c194c1be2 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:35.469670 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f810e16-cc9e-4610-ae42-483c194c1be2 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:35.469950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f810e16-cc9e-4610-ae42-483c194c1be2 None admin] PATH_INFO: `/v3/users/a0d9b4b095d14ded9dcba61cbc06b771` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:35.484211 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab35d22-3fd0-4152-83bc-ee5425f37f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:35.484683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f810e16-cc9e-4610-ae42-483c194c1be2 None admin] RBAC: Authorizing `identity:get_user(user_id=a0d9b4b095d14ded9dcba61cbc06b771)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:35.484914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab35d22-3fd0-4152-83bc-ee5425f37f89 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:35.485207 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab35d22-3fd0-4152-83bc-ee5425f37f89 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:35.485465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab35d22-3fd0-4152-83bc-ee5425f37f89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:35.486263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f810e16-cc9e-4610-ae42-483c194c1be2 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:35.488810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 95/198] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:35 2022] GET /identity/v3/users/a0d9b4b095d14ded9dcba61cbc06b771 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:35.489702 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f810e16-cc9e-4610-ae42-483c194c1be2 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:35.547211 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:35.571637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab35d22-3fd0-4152-83bc-ee5425f37f89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:35.573247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab35d22-3fd0-4152-83bc-ee5425f37f89 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:35.593037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:35.625412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f810e16-cc9e-4610-ae42-483c194c1be2 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:35.669001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6c876-47e0-4620-a2ca-89b1a3bb7afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:35.669642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6c876-47e0-4620-a2ca-89b1a3bb7afb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:35.669921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6c876-47e0-4620-a2ca-89b1a3bb7afb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:35.670193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6c876-47e0-4620-a2ca-89b1a3bb7afb None admin] PATH_INFO: `/v3/users/a0d9b4b095d14ded9dcba61cbc06b771` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:35.671720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6c876-47e0-4620-a2ca-89b1a3bb7afb None admin] RBAC: Authorizing `identity:get_user(user_id=a0d9b4b095d14ded9dcba61cbc06b771)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:35.673503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6c876-47e0-4620-a2ca-89b1a3bb7afb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:35.675952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 96/199] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:35 2022] GET /identity/v3/users/a0d9b4b095d14ded9dcba61cbc06b771 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:35.676983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6c876-47e0-4620-a2ca-89b1a3bb7afb None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:35.708890 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 104/200] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:38:35.714049 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab35d22-3fd0-4152-83bc-ee5425f37f89 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:35.805710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6c876-47e0-4620-a2ca-89b1a3bb7afb None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:35.820526 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab35d22-3fd0-4152-83bc-ee5425f37f89 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:35.847171 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:35.852890 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb33482-42cc-4ce2-a196-e4ddf35fefdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:35.853928 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb33482-42cc-4ce2-a196-e4ddf35fefdb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:35.854205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb33482-42cc-4ce2-a196-e4ddf35fefdb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:35.854472 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb33482-42cc-4ce2-a196-e4ddf35fefdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:35.860253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a1b73-a87a-4e3b-95af-5abdbec47311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:35.861183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a1b73-a87a-4e3b-95af-5abdbec47311 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:35.861706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a1b73-a87a-4e3b-95af-5abdbec47311 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:35.862197 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a1b73-a87a-4e3b-95af-5abdbec47311 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:35.873455 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:35.949437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb33482-42cc-4ce2-a196-e4ddf35fefdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:35.951370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb33482-42cc-4ce2-a196-e4ddf35fefdb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:35.961067 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a1b73-a87a-4e3b-95af-5abdbec47311 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:35.963764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a1b73-a87a-4e3b-95af-5abdbec47311 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:36.072383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 97/201] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:38:36.076477 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb33482-42cc-4ce2-a196-e4ddf35fefdb None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:36.079347 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:36.107880 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 105/202] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:38:36.107880 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a1b73-a87a-4e3b-95af-5abdbec47311 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:36.137170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:36.197778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb33482-42cc-4ce2-a196-e4ddf35fefdb None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:36.222915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a1b73-a87a-4e3b-95af-5abdbec47311 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:36.237598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e211bf9-2499-4b97-9f25-4b7b2d27bc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:36.238242 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e211bf9-2499-4b97-9f25-4b7b2d27bc4e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:36.238519 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e211bf9-2499-4b97-9f25-4b7b2d27bc4e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:36.238794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e211bf9-2499-4b97-9f25-4b7b2d27bc4e None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:36.240666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e211bf9-2499-4b97-9f25-4b7b2d27bc4e None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:36.242366 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e211bf9-2499-4b97-9f25-4b7b2d27bc4e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:36.244983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 98/203] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:36 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:36.245763 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e211bf9-2499-4b97-9f25-4b7b2d27bc4e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:36.275585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6775c-b7b1-48d8-941e-1d9510d8eb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:36.276276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6775c-b7b1-48d8-941e-1d9510d8eb6c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:36.276558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6775c-b7b1-48d8-941e-1d9510d8eb6c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:36.276832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d6775c-b7b1-48d8-941e-1d9510d8eb6c None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:36.279007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6775c-b7b1-48d8-941e-1d9510d8eb6c None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:36.280837 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d6775c-b7b1-48d8-941e-1d9510d8eb6c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:36.282827 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 106/204] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:36 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:36.283680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6775c-b7b1-48d8-941e-1d9510d8eb6c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:36.345022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:36.381142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e211bf9-2499-4b97-9f25-4b7b2d27bc4e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:36.400758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:36.411354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d6775c-b7b1-48d8-941e-1d9510d8eb6c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:36.425167 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed7136-b350-4ad6-9d37-9ea1bd837fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:36.425893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed7136-b350-4ad6-9d37-9ea1bd837fc3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:36.426247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed7136-b350-4ad6-9d37-9ea1bd837fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:36.426619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed7136-b350-4ad6-9d37-9ea1bd837fc3 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:36.427806 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed7136-b350-4ad6-9d37-9ea1bd837fc3 None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:36.429319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed7136-b350-4ad6-9d37-9ea1bd837fc3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:36.431169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 99/205] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:36 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:36.432011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed7136-b350-4ad6-9d37-9ea1bd837fc3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:36.454233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ebcae-5c0f-4bf7-83d5-550ee3a42c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:36.455234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ebcae-5c0f-4bf7-83d5-550ee3a42c5f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:36.455524 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ebcae-5c0f-4bf7-83d5-550ee3a42c5f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:36.455801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ebcae-5c0f-4bf7-83d5-550ee3a42c5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:36.456832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ebcae-5c0f-4bf7-83d5-550ee3a42c5f None admin] RBAC: Adding query filter params (role.id=2235d4074114441ea4661c3fb936dcc3, scope.project.id=4681da4f1deb408293b2c948aec851ea, user.id=c2dcd642c2e24ebdacea43238f322e82) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:36.457143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ebcae-5c0f-4bf7-83d5-550ee3a42c5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:36.459138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ebcae-5c0f-4bf7-83d5-550ee3a42c5f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:36.469451 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 107/206] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:36 2022] GET /identity/v3/role_assignments?user.id=c2dcd642c2e24ebdacea43238f322e82&scope.project.id=4681da4f1deb408293b2c948aec851ea&role.id=2235d4074114441ea4661c3fb936dcc3 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:36.472276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ebcae-5c0f-4bf7-83d5-550ee3a42c5f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:36.598524 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ebcae-5c0f-4bf7-83d5-550ee3a42c5f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:36.614566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:36.638287 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa48da-8a1b-4faf-adb2-769cb1694fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:36.638901 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa48da-8a1b-4faf-adb2-769cb1694fde None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:36.639272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa48da-8a1b-4faf-adb2-769cb1694fde None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:36.639596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa48da-8a1b-4faf-adb2-769cb1694fde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:36.640353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa48da-8a1b-4faf-adb2-769cb1694fde None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=17924018ca584f3696136b838ab2fdc8, user.id=a0d9b4b095d14ded9dcba61cbc06b771) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:36.640701 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa48da-8a1b-4faf-adb2-769cb1694fde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:36.641975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed7136-b350-4ad6-9d37-9ea1bd837fc3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:36.642174 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa48da-8a1b-4faf-adb2-769cb1694fde None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:36.654946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 108/207] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:36 2022] GET /identity/v3/role_assignments?user.id=a0d9b4b095d14ded9dcba61cbc06b771&scope.project.id=17924018ca584f3696136b838ab2fdc8&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:36.718824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed631ba-a525-42bf-974d-bffa5fb66ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:36.719460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed631ba-a525-42bf-974d-bffa5fb66ef4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:36.719741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed631ba-a525-42bf-974d-bffa5fb66ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:36.720042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed631ba-a525-42bf-974d-bffa5fb66ef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:36.720791 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed631ba-a525-42bf-974d-bffa5fb66ef4 None admin] RBAC: Adding query filter params (role.id=fe3bbad209634f16b1fcb66d6d49f665, scope.project.id=17924018ca584f3696136b838ab2fdc8, user.id=a0d9b4b095d14ded9dcba61cbc06b771) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:36.721102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed631ba-a525-42bf-974d-bffa5fb66ef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:36.722593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed631ba-a525-42bf-974d-bffa5fb66ef4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:36.733554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 100/208] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:36 2022] GET /identity/v3/role_assignments?user.id=a0d9b4b095d14ded9dcba61cbc06b771&scope.project.id=17924018ca584f3696136b838ab2fdc8&role.id=fe3bbad209634f16b1fcb66d6d49f665 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:37.799172 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3419ca3-f399-418f-b30f-662171d51f70 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:37.799936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3419ca3-f399-418f-b30f-662171d51f70 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:37.799936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3419ca3-f399-418f-b30f-662171d51f70 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:37.801437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 109/209] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:37.806893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b6922-7491-4cbb-b17b-58648a0296bf None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:37.807410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b6922-7491-4cbb-b17b-58648a0296bf None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:37.807410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b6922-7491-4cbb-b17b-58648a0296bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:37.828291 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-830b6922-7491-4cbb-b17b-58648a0296bf None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:37.829475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-830b6922-7491-4cbb-b17b-58648a0296bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:37.842926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7d112-de99-431b-9313-c069231c9a8b None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:37.843205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7d112-de99-431b-9313-c069231c9a8b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:37.843458 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7d112-de99-431b-9313-c069231c9a8b None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:37.845030 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 110/210] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:38:37.854886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef43c13-c802-4e4c-99da-42c5f4b40af2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:37.854886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef43c13-c802-4e4c-99da-42c5f4b40af2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:37.854886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef43c13-c802-4e4c-99da-42c5f4b40af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:37.871798 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8ef43c13-c802-4e4c-99da-42c5f4b40af2 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:37.873327 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8ef43c13-c802-4e4c-99da-42c5f4b40af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:37.880789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:37.915925 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 101/211] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:37.956282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 111/212] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:38.193612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b6922-7491-4cbb-b17b-58648a0296bf None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:38.230539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef43c13-c802-4e4c-99da-42c5f4b40af2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:38.296280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b6922-7491-4cbb-b17b-58648a0296bf None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:38.332394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8aaeb-30a0-456d-ba65-dbc807eb1283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:38.333034 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8aaeb-30a0-456d-ba65-dbc807eb1283 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:38.333313 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8aaeb-30a0-456d-ba65-dbc807eb1283 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:38.333587 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8aaeb-30a0-456d-ba65-dbc807eb1283 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:38.344326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df8aaeb-30a0-456d-ba65-dbc807eb1283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-0df8aaeb-30a0-456d-ba65-dbc807eb1283 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:38.349081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:38.349982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:38.350779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:38.350779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:38.350779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:38.350779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:38.350779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:38.350779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:38.350779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:38.350779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:38.350779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:38.350779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:38.350779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:38.350779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:38.350779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 102/213] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:38.354567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef43c13-c802-4e4c-99da-42c5f4b40af2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:38.358526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8aaeb-30a0-456d-ba65-dbc807eb1283 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:38.366005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:38.394546 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:38.398134 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237a3f4-6075-4d34-a180-f6077549c7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:38.399200 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237a3f4-6075-4d34-a180-f6077549c7c9 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:38.400497 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237a3f4-6075-4d34-a180-f6077549c7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:38.401055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9237a3f4-6075-4d34-a180-f6077549c7c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:38.402507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237a3f4-6075-4d34-a180-f6077549c7c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:38.404883 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9237a3f4-6075-4d34-a180-f6077549c7c9 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:38.406709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 112/214] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:38.412836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237a3f4-6075-4d34-a180-f6077549c7c9 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:38.472071 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8aaeb-30a0-456d-ba65-dbc807eb1283 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:38.519022 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38d09b-db97-4082-83e1-aa137c29efa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:38.520235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38d09b-db97-4082-83e1-aa137c29efa2 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:38.521118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38d09b-db97-4082-83e1-aa137c29efa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:38.521568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38d09b-db97-4082-83e1-aa137c29efa2 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:38.522426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38d09b-db97-4082-83e1-aa137c29efa2 None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:38.522907 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38d09b-db97-4082-83e1-aa137c29efa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:38.525054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38d09b-db97-4082-83e1-aa137c29efa2 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:38.540143 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 103/215] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:38.541287 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9237a3f4-6075-4d34-a180-f6077549c7c9 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:38.542537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38d09b-db97-4082-83e1-aa137c29efa2 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:38.581317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806ae48-2f4f-49ba-a02f-a742a4e66c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:38.582232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806ae48-2f4f-49ba-a02f-a742a4e66c27 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:38.582708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806ae48-2f4f-49ba-a02f-a742a4e66c27 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:38.583158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806ae48-2f4f-49ba-a02f-a742a4e66c27 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:38.584009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806ae48-2f4f-49ba-a02f-a742a4e66c27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:38.585648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806ae48-2f4f-49ba-a02f-a742a4e66c27 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:38.612326 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 113/216] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:38:38 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:38:38.664870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38d09b-db97-4082-83e1-aa137c29efa2 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:38.666960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:38.704840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c0e3cf-2c54-4e50-a37e-b1a42a01ec15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:38.705801 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0e3cf-2c54-4e50-a37e-b1a42a01ec15 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:38.706257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0e3cf-2c54-4e50-a37e-b1a42a01ec15 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:38.706737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c0e3cf-2c54-4e50-a37e-b1a42a01ec15 None admin] PATH_INFO: `/v3/users/7c6eec8c701842aaa94857500668d3db` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:38.724076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0e3cf-2c54-4e50-a37e-b1a42a01ec15 None admin] RBAC: Authorizing `identity:get_user(user_id=7c6eec8c701842aaa94857500668d3db)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:38.725987 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c0e3cf-2c54-4e50-a37e-b1a42a01ec15 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:38.728054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 104/217] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:38 2022] GET /identity/v3/users/7c6eec8c701842aaa94857500668d3db => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:38.739217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806ae48-2f4f-49ba-a02f-a742a4e66c27 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:38.782431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:38.874479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806ae48-2f4f-49ba-a02f-a742a4e66c27 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:38.916733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76781b0-3003-4d4f-b28a-d2ef02fc6831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:38.917378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76781b0-3003-4d4f-b28a-d2ef02fc6831 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:38.917663 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76781b0-3003-4d4f-b28a-d2ef02fc6831 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:38.917936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76781b0-3003-4d4f-b28a-d2ef02fc6831 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:38.920629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e3324-b489-4f08-afa6-00adab399fc8 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:38.921349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e3324-b489-4f08-afa6-00adab399fc8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:38.921422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e3324-b489-4f08-afa6-00adab399fc8 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:38.922775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 105/218] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:38.928408 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8d3be-a998-45f9-a273-01b181d58921 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:38.928694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8d3be-a998-45f9-a273-01b181d58921 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:38.928976 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8d3be-a998-45f9-a273-01b181d58921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:38.953141 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-dba8d3be-a998-45f9-a273-01b181d58921 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:38.954490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dba8d3be-a998-45f9-a273-01b181d58921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:38.999584 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76781b0-3003-4d4f-b28a-d2ef02fc6831 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:39.001098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76781b0-3003-4d4f-b28a-d2ef02fc6831 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:39.050169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 106/219] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:39.051165 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:39.051519 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9bce9-2029-4fc8-bca5-9cd32d652c12 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:39.051811 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9bce9-2029-4fc8-bca5-9cd32d652c12 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:39.052166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9bce9-2029-4fc8-bca5-9cd32d652c12 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:39.053444 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 107/220] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:39.059759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef31a06-c34c-4c2e-b76d-9c96e108247d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:39.060146 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef31a06-c34c-4c2e-b76d-9c96e108247d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:39.060422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef31a06-c34c-4c2e-b76d-9c96e108247d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:39.092143 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6ef31a06-c34c-4c2e-b76d-9c96e108247d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:39.093536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6ef31a06-c34c-4c2e-b76d-9c96e108247d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:39.137761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 114/221] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:38:39.143367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76781b0-3003-4d4f-b28a-d2ef02fc6831 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:39.175950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 108/222] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:39.258679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76781b0-3003-4d4f-b28a-d2ef02fc6831 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:39.292060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:39.304656 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264771fc-e397-441b-9f7a-d871c0a2cec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:39.305307 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264771fc-e397-441b-9f7a-d871c0a2cec7 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:39.305591 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264771fc-e397-441b-9f7a-d871c0a2cec7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:39.305867 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264771fc-e397-441b-9f7a-d871c0a2cec7 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:39.307118 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264771fc-e397-441b-9f7a-d871c0a2cec7 None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:39.308600 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264771fc-e397-441b-9f7a-d871c0a2cec7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:39.310629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 115/223] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:39 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:39.316761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef31a06-c34c-4c2e-b76d-9c96e108247d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:39.349701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:39.401787 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264771fc-e397-441b-9f7a-d871c0a2cec7 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:39.485062 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef31a06-c34c-4c2e-b76d-9c96e108247d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:39.524860 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0fc17-8efa-4ca9-b333-3d56552b26e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:39.525483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0fc17-8efa-4ca9-b333-3d56552b26e6 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:39.525767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0fc17-8efa-4ca9-b333-3d56552b26e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:39.526043 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0fc17-8efa-4ca9-b333-3d56552b26e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:39.526779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0fc17-8efa-4ca9-b333-3d56552b26e6 None admin] RBAC: Adding query filter params (role.id=0a6592b9a0b64df6938aba0a010c2256, scope.project.id=17924018ca584f3696136b838ab2fdc8, user.id=7c6eec8c701842aaa94857500668d3db) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:39.527089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0fc17-8efa-4ca9-b333-3d56552b26e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:39.529101 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f0fc17-8efa-4ca9-b333-3d56552b26e6 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:39.535547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 109/224] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:39 2022] GET /identity/v3/role_assignments?user.id=7c6eec8c701842aaa94857500668d3db&scope.project.id=17924018ca584f3696136b838ab2fdc8&role.id=0a6592b9a0b64df6938aba0a010c2256 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:39.541743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0fc17-8efa-4ca9-b333-3d56552b26e6 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:39.641752 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264771fc-e397-441b-9f7a-d871c0a2cec7 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:39.699009 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:39.717238 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16828659-7442-4a05-a377-a28848ca1219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:39.717901 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16828659-7442-4a05-a377-a28848ca1219 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:39.718240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16828659-7442-4a05-a377-a28848ca1219 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:39.718522 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16828659-7442-4a05-a377-a28848ca1219 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:39.735636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16828659-7442-4a05-a377-a28848ca1219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-16828659-7442-4a05-a377-a28848ca1219 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:39.740677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:39.741629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:39.742431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:39.742431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:39.742431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:39.742431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:39.742431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:39.742431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:39.742431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:39.742431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:39.742431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:39.742431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:39.742431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:39.742431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:39.742431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 116/225] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:38:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:38:39.749237 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f0fc17-8efa-4ca9-b333-3d56552b26e6 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:39.756838 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16828659-7442-4a05-a377-a28848ca1219 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:39.799160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1c98-b0c8-4575-a3a2-db3a2476d000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:39.800205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1c98-b0c8-4575-a3a2-db3a2476d000 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:39.800732 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1c98-b0c8-4575-a3a2-db3a2476d000 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:39.801213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ca1c98-b0c8-4575-a3a2-db3a2476d000 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:39.814526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ca1c98-b0c8-4575-a3a2-db3a2476d000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-43ca1c98-b0c8-4575-a3a2-db3a2476d000 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:39.820968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:39.821940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:39.823222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:39.823222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:39.823222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:39.823222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:39.823222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:39.823222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:39.823222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:39.823222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:39.823222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:39.823222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:39.823222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:38:39.823222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:39.825949 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 110/226] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:38:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:39.833654 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1c98-b0c8-4575-a3a2-db3a2476d000 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:39.896270 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:39.972338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16828659-7442-4a05-a377-a28848ca1219 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:39.988736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ca1c98-b0c8-4575-a3a2-db3a2476d000 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:40.045380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539e781-0bf7-4468-b132-8c735b16415b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:40.046406 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539e781-0bf7-4468-b132-8c735b16415b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:40.046681 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539e781-0bf7-4468-b132-8c735b16415b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:40.046946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539e781-0bf7-4468-b132-8c735b16415b None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:40.047631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539e781-0bf7-4468-b132-8c735b16415b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:40.047960 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539e781-0bf7-4468-b132-8c735b16415b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:40.050180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539e781-0bf7-4468-b132-8c735b16415b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:40.067317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 117/227] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:40.070380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dfc21-67f8-4733-802b-06121dc11e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:40.071005 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dfc21-67f8-4733-802b-06121dc11e8f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:40.071281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dfc21-67f8-4733-802b-06121dc11e8f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:40.071551 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dfc21-67f8-4733-802b-06121dc11e8f None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:40.072321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dfc21-67f8-4733-802b-06121dc11e8f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:40.072618 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dfc21-67f8-4733-802b-06121dc11e8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:40.076983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dfc21-67f8-4733-802b-06121dc11e8f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:40.078556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539e781-0bf7-4468-b132-8c735b16415b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:40.101727 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 111/228] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:38:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:40.104427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dfc21-67f8-4733-802b-06121dc11e8f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:40.108434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:40.246433 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539e781-0bf7-4468-b132-8c735b16415b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:40.258030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dfc21-67f8-4733-802b-06121dc11e8f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:40.269888 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:40.286796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2e632-d28c-4c06-bd23-21239385b5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:40.288469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2e632-d28c-4c06-bd23-21239385b5e1 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:40.289108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2e632-d28c-4c06-bd23-21239385b5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:40.290693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2e632-d28c-4c06-bd23-21239385b5e1 None admin] PATH_INFO: `/v3/users/a0d9b4b095d14ded9dcba61cbc06b771` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:40.294587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2e632-d28c-4c06-bd23-21239385b5e1 None admin] RBAC: Authorizing `identity:get_user(user_id=a0d9b4b095d14ded9dcba61cbc06b771)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:40.296562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2e632-d28c-4c06-bd23-21239385b5e1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:40.299811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 118/229] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:40 2022] GET /identity/v3/users/a0d9b4b095d14ded9dcba61cbc06b771 => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:40.305707 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2e632-d28c-4c06-bd23-21239385b5e1 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:40.333311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0011b-3f68-4fe3-a1db-5278664bc77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:40.333967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0011b-3f68-4fe3-a1db-5278664bc77e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:40.334247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0011b-3f68-4fe3-a1db-5278664bc77e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:40.334516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0011b-3f68-4fe3-a1db-5278664bc77e None admin] PATH_INFO: `/v3/users/a0d9b4b095d14ded9dcba61cbc06b771` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:40.336628 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0011b-3f68-4fe3-a1db-5278664bc77e None admin] RBAC: Authorizing `identity:get_user(user_id=a0d9b4b095d14ded9dcba61cbc06b771)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:40.338120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0011b-3f68-4fe3-a1db-5278664bc77e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:40.344433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 112/230] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:40 2022] GET /identity/v3/users/a0d9b4b095d14ded9dcba61cbc06b771 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:40.358615 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0011b-3f68-4fe3-a1db-5278664bc77e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:40.421382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2e632-d28c-4c06-bd23-21239385b5e1 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:40.456387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ccbef-7ceb-4d9b-8b5d-334f3f8a6aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:40.457392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ccbef-7ceb-4d9b-8b5d-334f3f8a6aa0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:40.457869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ccbef-7ceb-4d9b-8b5d-334f3f8a6aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:40.458342 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ccbef-7ceb-4d9b-8b5d-334f3f8a6aa0 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:40.459675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ccbef-7ceb-4d9b-8b5d-334f3f8a6aa0 None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:40.461345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ccbef-7ceb-4d9b-8b5d-334f3f8a6aa0 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:40.463533 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 119/231] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:40 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:40.471036 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:40.471232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ccbef-7ceb-4d9b-8b5d-334f3f8a6aa0 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:40.521891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0011b-3f68-4fe3-a1db-5278664bc77e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:40.568193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:40.586111 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29e572-724c-4341-ae60-fc3155528799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:40.586781 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29e572-724c-4341-ae60-fc3155528799 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:40.587065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29e572-724c-4341-ae60-fc3155528799 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:40.587339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d29e572-724c-4341-ae60-fc3155528799 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:40.592757 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29e572-724c-4341-ae60-fc3155528799 None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:40.594274 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d29e572-724c-4341-ae60-fc3155528799 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:40.596082 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 113/232] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:40 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:40.602391 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29e572-724c-4341-ae60-fc3155528799 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:40.628828 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ccbef-7ceb-4d9b-8b5d-334f3f8a6aa0 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:40.701905 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea9d8f-48c7-4368-ab80-5bf731dceadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:40.702572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea9d8f-48c7-4368-ab80-5bf731dceadb None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:40.705483 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea9d8f-48c7-4368-ab80-5bf731dceadb None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:40.705774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea9d8f-48c7-4368-ab80-5bf731dceadb None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8/users/a0d9b4b095d14ded9dcba61cbc06b771/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:40.708580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea9d8f-48c7-4368-ab80-5bf731dceadb None admin] RBAC: Authorizing `identity:create_grant(project_id=17924018ca584f3696136b838ab2fdc8, user_id=a0d9b4b095d14ded9dcba61cbc06b771, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:40.710019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea9d8f-48c7-4368-ab80-5bf731dceadb None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:40.718227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d29e572-724c-4341-ae60-fc3155528799 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:40.734863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 120/233] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:38:40 2022] PUT /identity/v3/projects/17924018ca584f3696136b838ab2fdc8/users/a0d9b4b095d14ded9dcba61cbc06b771/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:38:40.767893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b755998-54ca-433e-808f-a014c79d745b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:40.768573 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b755998-54ca-433e-808f-a014c79d745b None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:40.768853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b755998-54ca-433e-808f-a014c79d745b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:40.769186 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b755998-54ca-433e-808f-a014c79d745b None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8/users/a0d9b4b095d14ded9dcba61cbc06b771/roles/fe3bbad209634f16b1fcb66d6d49f665` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:40.771563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b755998-54ca-433e-808f-a014c79d745b None admin] RBAC: Authorizing `identity:create_grant(project_id=17924018ca584f3696136b838ab2fdc8, user_id=a0d9b4b095d14ded9dcba61cbc06b771, role_id=fe3bbad209634f16b1fcb66d6d49f665)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:40.773216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b755998-54ca-433e-808f-a014c79d745b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:40.789214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:40.799996 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 114/234] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:38:40 2022] PUT /identity/v3/projects/17924018ca584f3696136b838ab2fdc8/users/a0d9b4b095d14ded9dcba61cbc06b771/roles/fe3bbad209634f16b1fcb66d6d49f665 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:38:41.517661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1682ab-f395-477c-85d7-fa4776c57813 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:41.518178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1682ab-f395-477c-85d7-fa4776c57813 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:41.518178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1682ab-f395-477c-85d7-fa4776c57813 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:41.519785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 121/235] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:41.525311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68c594-abf3-4090-810c-f43c3ff28f26 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:41.525759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68c594-abf3-4090-810c-f43c3ff28f26 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:41.525936 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d68c594-abf3-4090-810c-f43c3ff28f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:41.556856 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8d68c594-abf3-4090-810c-f43c3ff28f26 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:41.558054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8d68c594-abf3-4090-810c-f43c3ff28f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:41.646190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 115/236] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:41.677684 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70859b-1c9a-4ffd-9e1f-74bccfa76938 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:41.678022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70859b-1c9a-4ffd-9e1f-74bccfa76938 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:41.678302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70859b-1c9a-4ffd-9e1f-74bccfa76938 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:41.680058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 122/237] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:41.685319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbacc9f-6d86-44e8-a43b-f7627550a398 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:41.685886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbacc9f-6d86-44e8-a43b-f7627550a398 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:41.686365 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbacc9f-6d86-44e8-a43b-f7627550a398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:41.708486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ecbacc9f-6d86-44e8-a43b-f7627550a398 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:41.709997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ecbacc9f-6d86-44e8-a43b-f7627550a398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:41.814586 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 116/238] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:41.957239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70859b-1c9a-4ffd-9e1f-74bccfa76938 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:42.079530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70859b-1c9a-4ffd-9e1f-74bccfa76938 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:42.101816 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:42.119158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd2152-1932-4453-9927-9eebf02fb5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:42.120569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd2152-1932-4453-9927-9eebf02fb5b0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:42.120854 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd2152-1932-4453-9927-9eebf02fb5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:42.121131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd2152-1932-4453-9927-9eebf02fb5b0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:42.134931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd2152-1932-4453-9927-9eebf02fb5b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-96dd2152-1932-4453-9927-9eebf02fb5b0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:42.140612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:42.141574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:42.142383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:42.142383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:42.142383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:42.142383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:42.142383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:42.142383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:42.142383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:42.142383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:42.142383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:42.142383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:42.142383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:42.142383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:42.142383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 123/239] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:42.148004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbacc9f-6d86-44e8-a43b-f7627550a398 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:42.221041 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:42.237435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5f878-1037-44e7-9558-6ed5c66c3f15 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:42.238035 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5f878-1037-44e7-9558-6ed5c66c3f15 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:42.238550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5f878-1037-44e7-9558-6ed5c66c3f15 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:42.240378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 124/240] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:38:42.245064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c532c-a6bb-4066-b483-dc0f6ec70dca None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:42.245574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c532c-a6bb-4066-b483-dc0f6ec70dca None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:42.246048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c532c-a6bb-4066-b483-dc0f6ec70dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:42.268541 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbacc9f-6d86-44e8-a43b-f7627550a398 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:42.284801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-fe1c532c-a6bb-4066-b483-dc0f6ec70dca None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:42.286298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fe1c532c-a6bb-4066-b483-dc0f6ec70dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:42.310618 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa55795-b629-48e0-9065-bb08115f7172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:42.311230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa55795-b629-48e0-9065-bb08115f7172 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:42.311510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa55795-b629-48e0-9065-bb08115f7172 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:42.311781 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa55795-b629-48e0-9065-bb08115f7172 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:42.312481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa55795-b629-48e0-9065-bb08115f7172 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:42.312782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa55795-b629-48e0-9065-bb08115f7172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:42.314178 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa55795-b629-48e0-9065-bb08115f7172 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:42.324419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 117/241] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:42.324419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa55795-b629-48e0-9065-bb08115f7172 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:42.371057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 125/242] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:42.372350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c532c-a6bb-4066-b483-dc0f6ec70dca None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:42.485594 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c532c-a6bb-4066-b483-dc0f6ec70dca None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:42.497182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:42.503292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa55795-b629-48e0-9065-bb08115f7172 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:42.536152 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:42.552258 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb613a-496a-44bf-b0aa-72ba7cbbd8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:42.552900 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb613a-496a-44bf-b0aa-72ba7cbbd8d2 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:42.553077 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477261f-625a-43c1-813b-1069bf3a65fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:42.553529 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb613a-496a-44bf-b0aa-72ba7cbbd8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:42.553529 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb613a-496a-44bf-b0aa-72ba7cbbd8d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:42.553780 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477261f-625a-43c1-813b-1069bf3a65fc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:42.553989 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477261f-625a-43c1-813b-1069bf3a65fc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:42.554243 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477261f-625a-43c1-813b-1069bf3a65fc None admin] PATH_INFO: `/v3/users/eb9ecb752c264dfea0001d636d2a2a43` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:42.554584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb613a-496a-44bf-b0aa-72ba7cbbd8d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:42.556066 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb613a-496a-44bf-b0aa-72ba7cbbd8d2 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:42.557935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 118/243] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:42.563296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb613a-496a-44bf-b0aa-72ba7cbbd8d2 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:42.584557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477261f-625a-43c1-813b-1069bf3a65fc None admin] RBAC: Authorizing `identity:get_user(user_id=eb9ecb752c264dfea0001d636d2a2a43)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:42.586038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477261f-625a-43c1-813b-1069bf3a65fc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:42.588189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 126/244] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:42 2022] GET /identity/v3/users/eb9ecb752c264dfea0001d636d2a2a43 => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:42.594296 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477261f-625a-43c1-813b-1069bf3a65fc None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:42.695369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb613a-496a-44bf-b0aa-72ba7cbbd8d2 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:42.734285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2b822-1a9f-4892-8a8c-9cb787f1ea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:42.735007 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2b822-1a9f-4892-8a8c-9cb787f1ea36 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:42.735292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2b822-1a9f-4892-8a8c-9cb787f1ea36 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:42.735556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2b822-1a9f-4892-8a8c-9cb787f1ea36 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:42.736272 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce2b822-1a9f-4892-8a8c-9cb787f1ea36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:42.737850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce2b822-1a9f-4892-8a8c-9cb787f1ea36 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:42.754791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477261f-625a-43c1-813b-1069bf3a65fc None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:42.762875 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 119/245] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:38:42 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:38:42.763818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2b822-1a9f-4892-8a8c-9cb787f1ea36 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:42.793301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f8166-df4c-4d64-8260-1624f0786551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:42.794484 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f8166-df4c-4d64-8260-1624f0786551 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:42.794762 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f8166-df4c-4d64-8260-1624f0786551 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:42.795034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f8166-df4c-4d64-8260-1624f0786551 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:42.839328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:42.866891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:42.891134 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f8166-df4c-4d64-8260-1624f0786551 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:42.892837 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f8166-df4c-4d64-8260-1624f0786551 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:42.938019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2b822-1a9f-4892-8a8c-9cb787f1ea36 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:42.977829 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db50686-2c49-4722-a197-83d4e584e4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:42.978403 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db50686-2c49-4722-a197-83d4e584e4bb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:42.978682 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db50686-2c49-4722-a197-83d4e584e4bb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:42.978955 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db50686-2c49-4722-a197-83d4e584e4bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:42.991188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db50686-2c49-4722-a197-83d4e584e4bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-8db50686-2c49-4722-a197-83d4e584e4bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:42.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:42.998926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:42.999961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:42.999961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:42.999961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:42.999961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:42.999961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:42.999961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:42.999961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:42.999961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:42.999961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:42.999961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:42.999961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:42.999961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:42.999961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 120/246] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:43.017049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db50686-2c49-4722-a197-83d4e584e4bb None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:43.043107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 127/247] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:38:43.049397 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f8166-df4c-4d64-8260-1624f0786551 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:43.130207 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:43.138558 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db50686-2c49-4722-a197-83d4e584e4bb None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:43.145741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:43.170830 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f8166-df4c-4d64-8260-1624f0786551 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:43.176836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbd238-f5b1-4b20-babf-07e4a1d46cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:43.177135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbd238-f5b1-4b20-babf-07e4a1d46cc4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:43.177518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbd238-f5b1-4b20-babf-07e4a1d46cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:43.177892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbd238-f5b1-4b20-babf-07e4a1d46cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:43.178677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbd238-f5b1-4b20-babf-07e4a1d46cc4 None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:43.179079 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbd238-f5b1-4b20-babf-07e4a1d46cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:43.181854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbbd238-f5b1-4b20-babf-07e4a1d46cc4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:43.191731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 121/248] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:43.200893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbd238-f5b1-4b20-babf-07e4a1d46cc4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:43.234558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81f590-6ee4-48dc-a109-6bfdf52eae4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:43.235230 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81f590-6ee4-48dc-a109-6bfdf52eae4b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:43.235511 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81f590-6ee4-48dc-a109-6bfdf52eae4b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:43.235927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe81f590-6ee4-48dc-a109-6bfdf52eae4b None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:43.239607 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81f590-6ee4-48dc-a109-6bfdf52eae4b None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:43.241199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe81f590-6ee4-48dc-a109-6bfdf52eae4b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:43.242990 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 128/249] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:43 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:43.249143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81f590-6ee4-48dc-a109-6bfdf52eae4b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:43.329300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbd238-f5b1-4b20-babf-07e4a1d46cc4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:43.375009 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809d3f7-9937-4ed8-9790-834806510088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:43.375660 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809d3f7-9937-4ed8-9790-834806510088 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:43.376738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809d3f7-9937-4ed8-9790-834806510088 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:43.379139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809d3f7-9937-4ed8-9790-834806510088 None admin] PATH_INFO: `/v3/users/7c6eec8c701842aaa94857500668d3db` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:43.380924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809d3f7-9937-4ed8-9790-834806510088 None admin] RBAC: Authorizing `identity:get_user(user_id=7c6eec8c701842aaa94857500668d3db)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:43.382451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809d3f7-9937-4ed8-9790-834806510088 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:43.383794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 122/250] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:43 2022] GET /identity/v3/users/7c6eec8c701842aaa94857500668d3db => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:43.391361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809d3f7-9937-4ed8-9790-834806510088 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:43.411457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe81f590-6ee4-48dc-a109-6bfdf52eae4b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:43.445186 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:43.461444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846cd9d-ab6f-4e26-9c7e-6ee9890205a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:43.462038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846cd9d-ab6f-4e26-9c7e-6ee9890205a6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:43.462316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846cd9d-ab6f-4e26-9c7e-6ee9890205a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:43.462587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846cd9d-ab6f-4e26-9c7e-6ee9890205a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:43.463349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846cd9d-ab6f-4e26-9c7e-6ee9890205a6 None admin] RBAC: Adding query filter params (role.id=2235d4074114441ea4661c3fb936dcc3, scope.project.id=17924018ca584f3696136b838ab2fdc8, user.id=eb9ecb752c264dfea0001d636d2a2a43) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:43.463610 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846cd9d-ab6f-4e26-9c7e-6ee9890205a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:43.467151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5846cd9d-ab6f-4e26-9c7e-6ee9890205a6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:43.469893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:43.475907 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 129/251] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:43 2022] GET /identity/v3/role_assignments?user.id=eb9ecb752c264dfea0001d636d2a2a43&scope.project.id=17924018ca584f3696136b838ab2fdc8&role.id=2235d4074114441ea4661c3fb936dcc3 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:43.540923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809d3f7-9937-4ed8-9790-834806510088 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:43.587542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5a4f1-4846-4cb4-a4f5-b89adaa223c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:43.588875 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5a4f1-4846-4cb4-a4f5-b89adaa223c1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:43.589433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5a4f1-4846-4cb4-a4f5-b89adaa223c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:43.589939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5a4f1-4846-4cb4-a4f5-b89adaa223c1 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:43.591809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5a4f1-4846-4cb4-a4f5-b89adaa223c1 None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:43.593986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5a4f1-4846-4cb4-a4f5-b89adaa223c1 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:43.599916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 123/252] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:43 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:43.604261 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846cd9d-ab6f-4e26-9c7e-6ee9890205a6 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:43.730252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846cd9d-ab6f-4e26-9c7e-6ee9890205a6 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:43.734539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683a4ac-a677-45a2-82fa-c97614c2aa92 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:43.735098 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683a4ac-a677-45a2-82fa-c97614c2aa92 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:43.735579 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683a4ac-a677-45a2-82fa-c97614c2aa92 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:43.738527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 124/253] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:38:43.746437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebcd1e-27f4-4c35-9acb-c4fe6b067160 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:43.747539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebcd1e-27f4-4c35-9acb-c4fe6b067160 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:43.748036 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aebcd1e-27f4-4c35-9acb-c4fe6b067160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:43.776766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74edb0-58de-4d3c-a807-d7ddd0e8a9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:43.777335 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74edb0-58de-4d3c-a807-d7ddd0e8a9f0 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:43.777619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74edb0-58de-4d3c-a807-d7ddd0e8a9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:43.777914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74edb0-58de-4d3c-a807-d7ddd0e8a9f0 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8/users/7c6eec8c701842aaa94857500668d3db/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:43.778403 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8aebcd1e-27f4-4c35-9acb-c4fe6b067160 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:43.780104 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8aebcd1e-27f4-4c35-9acb-c4fe6b067160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:43.780209 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74edb0-58de-4d3c-a807-d7ddd0e8a9f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=17924018ca584f3696136b838ab2fdc8, user_id=7c6eec8c701842aaa94857500668d3db, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:43.781575 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74edb0-58de-4d3c-a807-d7ddd0e8a9f0 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:43.797785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 130/254] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:38:43 2022] PUT /identity/v3/projects/17924018ca584f3696136b838ab2fdc8/users/7c6eec8c701842aaa94857500668d3db/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:38:43.799078 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b63497-fcc9-49fe-9da6-4fab6d98e691 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:43.799370 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b63497-fcc9-49fe-9da6-4fab6d98e691 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:43.799725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b63497-fcc9-49fe-9da6-4fab6d98e691 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:43.801275 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 131/255] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:43.807174 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd1322-7b54-4226-85ab-a07cb0e004f2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:43.807357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd1322-7b54-4226-85ab-a07cb0e004f2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:43.807584 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd1322-7b54-4226-85ab-a07cb0e004f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:43.843870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3abd1322-7b54-4226-85ab-a07cb0e004f2 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:43.843870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3abd1322-7b54-4226-85ab-a07cb0e004f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:43.915097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:43.915479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 125/256] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:43.929229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 132/257] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:44.217163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebcd1e-27f4-4c35-9acb-c4fe6b067160 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:44.254743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:44.345754 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aebcd1e-27f4-4c35-9acb-c4fe6b067160 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:44.387318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4f19e-1a68-43d5-9662-5fab32b9efd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:44.388358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4f19e-1a68-43d5-9662-5fab32b9efd3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:44.388854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4f19e-1a68-43d5-9662-5fab32b9efd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:44.389319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4f19e-1a68-43d5-9662-5fab32b9efd3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:44.405830 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4f19e-1a68-43d5-9662-5fab32b9efd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-f1b4f19e-1a68-43d5-9662-5fab32b9efd3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:44.411029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:44.411932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:44.412730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:44.412730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:44.412730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:44.412730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:44.412730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:44.412730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:44.412730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:44.412730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:44.412730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:44.412730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:44.412730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:38:44.412730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:44.414985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 126/258] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:38:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:38:44.422091 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd1322-7b54-4226-85ab-a07cb0e004f2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:44.462175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4f19e-1a68-43d5-9662-5fab32b9efd3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:44.585903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:44.626487 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4f19e-1a68-43d5-9662-5fab32b9efd3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:44.693624 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f031a9-e43b-4781-ad38-5b6e171289dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:44.694290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f031a9-e43b-4781-ad38-5b6e171289dd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:44.694573 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f031a9-e43b-4781-ad38-5b6e171289dd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:44.694844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f031a9-e43b-4781-ad38-5b6e171289dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:44.719434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f031a9-e43b-4781-ad38-5b6e171289dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-49f031a9-e43b-4781-ad38-5b6e171289dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:44.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:44.729967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:44.732004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:44.732004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:44.732004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:44.732004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:44.732004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:44.732004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:44.732004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:44.732004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:44.732004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:44.732004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:44.732004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:38:44.732004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:44.732004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 127/259] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:38:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:38:44.732004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f031a9-e43b-4781-ad38-5b6e171289dd None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:44.781379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd1322-7b54-4226-85ab-a07cb0e004f2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:44.909779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f031a9-e43b-4781-ad38-5b6e171289dd None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:44.912752 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006d4c7-1c90-4d50-b609-1f4fd91963f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:44.913397 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006d4c7-1c90-4d50-b609-1f4fd91963f9 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:44.913687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006d4c7-1c90-4d50-b609-1f4fd91963f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:44.913944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8006d4c7-1c90-4d50-b609-1f4fd91963f9 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:44.914629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006d4c7-1c90-4d50-b609-1f4fd91963f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:44.914926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006d4c7-1c90-4d50-b609-1f4fd91963f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:44.916390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8006d4c7-1c90-4d50-b609-1f4fd91963f9 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:44.934844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 133/260] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:38:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 514 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:44.940226 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006d4c7-1c90-4d50-b609-1f4fd91963f9 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:44.969992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0ee68-9a17-4ed4-b3fa-304d594e53da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:44.970616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0ee68-9a17-4ed4-b3fa-304d594e53da None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:44.970895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0ee68-9a17-4ed4-b3fa-304d594e53da None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:44.971166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0ee68-9a17-4ed4-b3fa-304d594e53da None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:44.971858 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0ee68-9a17-4ed4-b3fa-304d594e53da None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:44.972140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0ee68-9a17-4ed4-b3fa-304d594e53da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:44.973556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0ee68-9a17-4ed4-b3fa-304d594e53da None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:44.989505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 128/261] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:45.000203 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0ee68-9a17-4ed4-b3fa-304d594e53da None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:45.030779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:45.080356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:45.227114 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8006d4c7-1c90-4d50-b609-1f4fd91963f9 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:45.231790 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0ee68-9a17-4ed4-b3fa-304d594e53da None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:45.298997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba0dd7-63b5-4996-85f1-f35ca4b62321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:45.300063 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba0dd7-63b5-4996-85f1-f35ca4b62321 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:45.300583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba0dd7-63b5-4996-85f1-f35ca4b62321 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:45.301057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba0dd7-63b5-4996-85f1-f35ca4b62321 None admin] PATH_INFO: `/v3/users/a0d9b4b095d14ded9dcba61cbc06b771` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:45.302771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba0dd7-63b5-4996-85f1-f35ca4b62321 None admin] RBAC: Authorizing `identity:get_user(user_id=a0d9b4b095d14ded9dcba61cbc06b771)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:45.304508 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ba0dd7-63b5-4996-85f1-f35ca4b62321 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:45.304654 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367ad04-0fa2-44f0-9874-8221150ac59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:45.305593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367ad04-0fa2-44f0-9874-8221150ac59c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:45.306043 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367ad04-0fa2-44f0-9874-8221150ac59c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:45.306451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367ad04-0fa2-44f0-9874-8221150ac59c None admin] PATH_INFO: `/v3/users/a0d9b4b095d14ded9dcba61cbc06b771` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:45.306692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 134/262] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:44 2022] GET /identity/v3/users/a0d9b4b095d14ded9dcba61cbc06b771 => generated 283 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:45.307990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367ad04-0fa2-44f0-9874-8221150ac59c None admin] RBAC: Authorizing `identity:get_user(user_id=a0d9b4b095d14ded9dcba61cbc06b771)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:45.310275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367ad04-0fa2-44f0-9874-8221150ac59c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:45.313130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba0dd7-63b5-4996-85f1-f35ca4b62321 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:45.313257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 129/263] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:44 2022] GET /identity/v3/users/a0d9b4b095d14ded9dcba61cbc06b771 => generated 283 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:45.318415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367ad04-0fa2-44f0-9874-8221150ac59c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:45.450360 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba0dd7-63b5-4996-85f1-f35ca4b62321 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:45.475698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:45.509479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367ad04-0fa2-44f0-9874-8221150ac59c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:45.509993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:45.554328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9897357-4787-425a-b327-6a52a6e17a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:45.554966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9897357-4787-425a-b327-6a52a6e17a80 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:45.555466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9897357-4787-425a-b327-6a52a6e17a80 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:45.555925 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9897357-4787-425a-b327-6a52a6e17a80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:45.592649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16de2c-c7a3-4ba5-9fdb-b351fc253d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:45.593958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16de2c-c7a3-4ba5-9fdb-b351fc253d41 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:45.594383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16de2c-c7a3-4ba5-9fdb-b351fc253d41 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:45.594826 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16de2c-c7a3-4ba5-9fdb-b351fc253d41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:45.687506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9897357-4787-425a-b327-6a52a6e17a80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:45.689233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9897357-4787-425a-b327-6a52a6e17a80 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:45.720638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16de2c-c7a3-4ba5-9fdb-b351fc253d41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:45.722367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16de2c-c7a3-4ba5-9fdb-b351fc253d41 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:45.887935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:45.888104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 135/264] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 14:38:45.894361 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9897357-4787-425a-b327-6a52a6e17a80 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:45.914715 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 130/265] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 14:38:45.920564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16de2c-c7a3-4ba5-9fdb-b351fc253d41 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:45.928314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:46.111615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9897357-4787-425a-b327-6a52a6e17a80 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:46.113236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16de2c-c7a3-4ba5-9fdb-b351fc253d41 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:46.177260 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb753b-0242-411c-b2b3-ea93b7418526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:46.177905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb753b-0242-411c-b2b3-ea93b7418526 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:46.179304 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517627b-00cc-4ac4-888d-1623324e396b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:46.179962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517627b-00cc-4ac4-888d-1623324e396b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:46.180242 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517627b-00cc-4ac4-888d-1623324e396b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:46.180514 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517627b-00cc-4ac4-888d-1623324e396b None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:46.181197 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb753b-0242-411c-b2b3-ea93b7418526 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:46.181480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb753b-0242-411c-b2b3-ea93b7418526 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:46.182642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517627b-00cc-4ac4-888d-1623324e396b None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:46.184113 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517627b-00cc-4ac4-888d-1623324e396b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:46.186466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 136/266] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:45 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:46.187131 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb753b-0242-411c-b2b3-ea93b7418526 None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:46.192712 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb753b-0242-411c-b2b3-ea93b7418526 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:46.194472 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 131/267] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:45 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:46.196357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517627b-00cc-4ac4-888d-1623324e396b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:46.200472 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb753b-0242-411c-b2b3-ea93b7418526 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:46.290123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:46.357478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:46.369096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb753b-0242-411c-b2b3-ea93b7418526 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:46.394658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517627b-00cc-4ac4-888d-1623324e396b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:46.416880 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8b970-875f-4d95-88d0-282e69960de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:46.417826 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8b970-875f-4d95-88d0-282e69960de6 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:46.418291 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8b970-875f-4d95-88d0-282e69960de6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:46.419306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8b970-875f-4d95-88d0-282e69960de6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:46.420271 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8b970-875f-4d95-88d0-282e69960de6 None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=17924018ca584f3696136b838ab2fdc8, user.id=a0d9b4b095d14ded9dcba61cbc06b771) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:46.420768 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8b970-875f-4d95-88d0-282e69960de6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:46.423506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8b970-875f-4d95-88d0-282e69960de6 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:46.437505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 132/268] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:46 2022] GET /identity/v3/role_assignments?user.id=a0d9b4b095d14ded9dcba61cbc06b771&scope.project.id=17924018ca584f3696136b838ab2fdc8&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:46.516716 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46be49c-b380-4537-b35c-befc8afc78fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:46.517359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46be49c-b380-4537-b35c-befc8afc78fb None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:46.517641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46be49c-b380-4537-b35c-befc8afc78fb None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:46.517911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46be49c-b380-4537-b35c-befc8afc78fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:46.518647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46be49c-b380-4537-b35c-befc8afc78fb None admin] RBAC: Adding query filter params (role.id=fe3bbad209634f16b1fcb66d6d49f665, scope.project.id=17924018ca584f3696136b838ab2fdc8, user.id=a0d9b4b095d14ded9dcba61cbc06b771) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:46.518951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46be49c-b380-4537-b35c-befc8afc78fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:46.520480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46be49c-b380-4537-b35c-befc8afc78fb None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:46.533175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 137/269] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:46 2022] GET /identity/v3/role_assignments?user.id=a0d9b4b095d14ded9dcba61cbc06b771&scope.project.id=17924018ca584f3696136b838ab2fdc8&role.id=fe3bbad209634f16b1fcb66d6d49f665 => generated 608 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:46.737384 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2312a-928b-41be-959b-41683c0ce5a7 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:46.737892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2312a-928b-41be-959b-41683c0ce5a7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:46.737892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2312a-928b-41be-959b-41683c0ce5a7 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:46.739506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 133/270] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:46.754419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9774a-16b4-48ef-8ad9-fed3471014fd None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:46.754726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9774a-16b4-48ef-8ad9-fed3471014fd None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:46.754976 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9774a-16b4-48ef-8ad9-fed3471014fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:46.785137 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-80c9774a-16b4-48ef-8ad9-fed3471014fd None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:46.786405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-80c9774a-16b4-48ef-8ad9-fed3471014fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:46.817732 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1167ea-c940-4e56-a6a3-1922703a3ce6 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:46.818035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1167ea-c940-4e56-a6a3-1922703a3ce6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:46.818281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1167ea-c940-4e56-a6a3-1922703a3ce6 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:46.819620 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 134/271] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:46.825492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f87d4-f6f7-43bc-b07a-759774ac0af8 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:46.825775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f87d4-f6f7-43bc-b07a-759774ac0af8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:46.826046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f87d4-f6f7-43bc-b07a-759774ac0af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:46.849162 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3c0f87d4-f6f7-43bc-b07a-759774ac0af8 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:46.850358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3c0f87d4-f6f7-43bc-b07a-759774ac0af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:46.910466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 138/272] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:46.935639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 135/273] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:47.013165 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776da7f0-24ff-4449-8c7e-d0feb688f290 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:47.013456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776da7f0-24ff-4449-8c7e-d0feb688f290 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:47.013636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776da7f0-24ff-4449-8c7e-d0feb688f290 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:47.015261 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 139/274] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:47.020806 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a33d46-d6c6-4d2a-b759-63263463fdfc None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:47.021124 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a33d46-d6c6-4d2a-b759-63263463fdfc None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:47.021356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a33d46-d6c6-4d2a-b759-63263463fdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:47.041104 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-75a33d46-d6c6-4d2a-b759-63263463fdfc None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:47.042285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-75a33d46-d6c6-4d2a-b759-63263463fdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:47.100840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:47.128511 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d859244-0757-48f8-901a-7fdf0e7ff5ea None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:47.128816 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d859244-0757-48f8-901a-7fdf0e7ff5ea None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:47.129068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d859244-0757-48f8-901a-7fdf0e7ff5ea None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:47.130624 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 140/275] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:47.141910 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 136/276] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:47.148591 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d98528-22f7-4ffe-a1b3-a47c8417add1 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:47.148877 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d98528-22f7-4ffe-a1b3-a47c8417add1 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:47.149147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d98528-22f7-4ffe-a1b3-a47c8417add1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:47.167833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e5d98528-22f7-4ffe-a1b3-a47c8417add1 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:47.169549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e5d98528-22f7-4ffe-a1b3-a47c8417add1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:47.233014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a33d46-d6c6-4d2a-b759-63263463fdfc None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:47.249435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 141/277] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:47.250340 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d98528-22f7-4ffe-a1b3-a47c8417add1 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:47.263595 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:47.362227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d98528-22f7-4ffe-a1b3-a47c8417add1 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:47.369576 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a33d46-d6c6-4d2a-b759-63263463fdfc None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:47.397679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf532d28-d1ab-4d23-8c8a-c0cb09e86298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:47.398309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf532d28-d1ab-4d23-8c8a-c0cb09e86298 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:47.398585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf532d28-d1ab-4d23-8c8a-c0cb09e86298 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:47.398852 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf532d28-d1ab-4d23-8c8a-c0cb09e86298 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:47.410032 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf532d28-d1ab-4d23-8c8a-c0cb09e86298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:47.414625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc18c07-fd84-4e7b-955c-12a471cdd5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:47.415272 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc18c07-fd84-4e7b-955c-12a471cdd5d8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:47.415556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc18c07-fd84-4e7b-955c-12a471cdd5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-bf532d28-d1ab-4d23-8c8a-c0cb09e86298 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:47.415764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:47.416648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:47.417456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:47.417456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:47.417456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:47.417456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:47.417456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:47.417456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:47.417456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:47.417456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:47.417456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:47.417456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:47.417456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:47.417456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:47.418071 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc18c07-fd84-4e7b-955c-12a471cdd5d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:47.418071 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc18c07-fd84-4e7b-955c-12a471cdd5d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:47.418202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 142/278] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:47.418202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf532d28-d1ab-4d23-8c8a-c0cb09e86298 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:47.418881 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc18c07-fd84-4e7b-955c-12a471cdd5d8 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:47.423868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 137/279] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:47.423868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc18c07-fd84-4e7b-955c-12a471cdd5d8 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:47.469798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:47.499394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:47.531083 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc18c07-fd84-4e7b-955c-12a471cdd5d8 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:47.535075 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf532d28-d1ab-4d23-8c8a-c0cb09e86298 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:47.565203 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ece472-ecdc-4666-adc0-2dc605e90f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:47.565833 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ece472-ecdc-4666-adc0-2dc605e90f35 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:47.566074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ece472-ecdc-4666-adc0-2dc605e90f35 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:47.566344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ece472-ecdc-4666-adc0-2dc605e90f35 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:47.567020 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ece472-ecdc-4666-adc0-2dc605e90f35 None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:47.567320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ece472-ecdc-4666-adc0-2dc605e90f35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:47.568630 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c358c-3b62-4645-889d-bf66d4a9fe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:47.568837 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ece472-ecdc-4666-adc0-2dc605e90f35 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:47.569257 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c358c-3b62-4645-889d-bf66d4a9fe4e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:47.569538 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c358c-3b62-4645-889d-bf66d4a9fe4e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:47.569931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c358c-3b62-4645-889d-bf66d4a9fe4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:47.577654 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 138/280] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:47.578445 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ece472-ecdc-4666-adc0-2dc605e90f35 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:47.582468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c358c-3b62-4645-889d-bf66d4a9fe4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-b67c358c-3b62-4645-889d-bf66d4a9fe4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:47.587285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:47.588379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:47.589181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:47.589181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:47.589181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:47.589181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:47.589181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:47.589181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:47.589181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:47.589181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:47.589181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:47.589181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:47.589181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:47.589181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:47.589181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 143/281] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:47.589181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c358c-3b62-4645-889d-bf66d4a9fe4e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:47.682375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ece472-ecdc-4666-adc0-2dc605e90f35 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:47.686248 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:47.690607 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c358c-3b62-4645-889d-bf66d4a9fe4e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:47.718379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17733b6-86b6-458b-8a51-3872b3e37413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:47.718608 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:47.718984 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17733b6-86b6-458b-8a51-3872b3e37413 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:47.719260 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17733b6-86b6-458b-8a51-3872b3e37413 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:47.719526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17733b6-86b6-458b-8a51-3872b3e37413 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:47.720536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17733b6-86b6-458b-8a51-3872b3e37413 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:47.721972 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17733b6-86b6-458b-8a51-3872b3e37413 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:47.724570 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb5e8d-b79c-458d-ac49-fe7592dc8a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:47.725197 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb5e8d-b79c-458d-ac49-fe7592dc8a3a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:47.725478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb5e8d-b79c-458d-ac49-fe7592dc8a3a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:47.725752 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb5e8d-b79c-458d-ac49-fe7592dc8a3a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:47.737328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb5e8d-b79c-458d-ac49-fe7592dc8a3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-a1cb5e8d-b79c-458d-ac49-fe7592dc8a3a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:47.743419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:47.744383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:47.748702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:47.748702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:47.748702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:47.748702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:47.748702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:47.748702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:47.748702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:47.748702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:47.748702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:47.748702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:47.748702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:47.748702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:47.748702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 144/282] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:47.748702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb5e8d-b79c-458d-ac49-fe7592dc8a3a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:47.749415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 139/283] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:38:47 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:38:47.749415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17733b6-86b6-458b-8a51-3872b3e37413 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:47.876365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb5e8d-b79c-458d-ac49-fe7592dc8a3a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:47.878315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17733b6-86b6-458b-8a51-3872b3e37413 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:47.916497 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d987d-579e-4b19-a2b7-96ee39a8359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:47.917146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d987d-579e-4b19-a2b7-96ee39a8359c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:47.917422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d987d-579e-4b19-a2b7-96ee39a8359c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:47.917650 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0cfaf-d194-42b3-b0e1-3c90d93d75a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:47.917765 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d987d-579e-4b19-a2b7-96ee39a8359c None admin] PATH_INFO: `/v3/users/4f39adfbcf654dd6a0900eb89af6fab8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:47.918334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0cfaf-d194-42b3-b0e1-3c90d93d75a6 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:47.918644 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0cfaf-d194-42b3-b0e1-3c90d93d75a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:47.918951 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f0cfaf-d194-42b3-b0e1-3c90d93d75a6 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:47.919629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0cfaf-d194-42b3-b0e1-3c90d93d75a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:47.919982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0cfaf-d194-42b3-b0e1-3c90d93d75a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:47.921633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f0cfaf-d194-42b3-b0e1-3c90d93d75a6 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:47.934601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d987d-579e-4b19-a2b7-96ee39a8359c None admin] RBAC: Authorizing `identity:get_user(user_id=4f39adfbcf654dd6a0900eb89af6fab8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:47.936511 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 140/284] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:47.936641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d987d-579e-4b19-a2b7-96ee39a8359c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:47.937864 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0cfaf-d194-42b3-b0e1-3c90d93d75a6 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:47.938096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 145/285] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:47 2022] GET /identity/v3/users/4f39adfbcf654dd6a0900eb89af6fab8 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:47.943972 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d987d-579e-4b19-a2b7-96ee39a8359c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:47.966154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:48.002985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:48.058791 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f0cfaf-d194-42b3-b0e1-3c90d93d75a6 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:48.063983 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d987d-579e-4b19-a2b7-96ee39a8359c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:48.105390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d2412-0182-4fbc-8ac7-877813d4d3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:48.106015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d2412-0182-4fbc-8ac7-877813d4d3f1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:48.106696 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c891e-7d6c-4b38-a17b-8d005364ef42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:48.106805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d2412-0182-4fbc-8ac7-877813d4d3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:48.107082 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d2412-0182-4fbc-8ac7-877813d4d3f1 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:48.107289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c891e-7d6c-4b38-a17b-8d005364ef42 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:48.107566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c891e-7d6c-4b38-a17b-8d005364ef42 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:48.107751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d2412-0182-4fbc-8ac7-877813d4d3f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:48.107864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c891e-7d6c-4b38-a17b-8d005364ef42 None admin] PATH_INFO: `/v3/users/7c6eec8c701842aaa94857500668d3db` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:48.108104 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d2412-0182-4fbc-8ac7-877813d4d3f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:48.108980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c891e-7d6c-4b38-a17b-8d005364ef42 None admin] RBAC: Authorizing `identity:get_user(user_id=7c6eec8c701842aaa94857500668d3db)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:48.109534 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585d2412-0182-4fbc-8ac7-877813d4d3f1 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:48.110400 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c891e-7d6c-4b38-a17b-8d005364ef42 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:48.112263 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 146/286] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:47 2022] GET /identity/v3/users/7c6eec8c701842aaa94857500668d3db => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:48.112807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c891e-7d6c-4b38-a17b-8d005364ef42 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:48.121395 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 141/287] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:48.122822 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d2412-0182-4fbc-8ac7-877813d4d3f1 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:48.224524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:48.245082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c891e-7d6c-4b38-a17b-8d005364ef42 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:48.249951 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585d2412-0182-4fbc-8ac7-877813d4d3f1 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:48.263394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:48.286462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91259dc-64ee-44e8-9508-33cb98720874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:48.287078 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91259dc-64ee-44e8-9508-33cb98720874 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:48.287380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91259dc-64ee-44e8-9508-33cb98720874 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:48.287859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91259dc-64ee-44e8-9508-33cb98720874 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:48.288503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91259dc-64ee-44e8-9508-33cb98720874 None admin] RBAC: Adding query filter params (role.id=0a6592b9a0b64df6938aba0a010c2256, user.id=4f39adfbcf654dd6a0900eb89af6fab8, scope.system=all) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:48.288814 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91259dc-64ee-44e8-9508-33cb98720874 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:48.290244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91259dc-64ee-44e8-9508-33cb98720874 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:48.294361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec796e-1064-4e0c-a842-a315013da6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:48.295300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec796e-1064-4e0c-a842-a315013da6f0 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:48.295750 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec796e-1064-4e0c-a842-a315013da6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:48.296237 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec796e-1064-4e0c-a842-a315013da6f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:48.296948 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 147/288] 10.222.0.60 () {56 vars in 1308 bytes} [Mon Sep 19 14:38:48 2022] GET /identity/v3/role_assignments?user.id=4f39adfbcf654dd6a0900eb89af6fab8&scope.system=all&role.id=0a6592b9a0b64df6938aba0a010c2256 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:48.297926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91259dc-64ee-44e8-9508-33cb98720874 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:48.386157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec796e-1064-4e0c-a842-a315013da6f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:48.388969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec796e-1064-4e0c-a842-a315013da6f0 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:48.414421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91259dc-64ee-44e8-9508-33cb98720874 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:48.458846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6088d4-51fb-4125-9d3b-422d6983cb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:48.459608 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6088d4-51fb-4125-9d3b-422d6983cb6b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:48.460017 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6088d4-51fb-4125-9d3b-422d6983cb6b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:48.460362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6088d4-51fb-4125-9d3b-422d6983cb6b None admin] PATH_INFO: `/v3/users/eb9ecb752c264dfea0001d636d2a2a43` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:48.461653 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6088d4-51fb-4125-9d3b-422d6983cb6b None admin] RBAC: Authorizing `identity:get_user(user_id=eb9ecb752c264dfea0001d636d2a2a43)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:48.463305 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6088d4-51fb-4125-9d3b-422d6983cb6b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:48.465156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 148/289] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:48 2022] GET /identity/v3/users/eb9ecb752c264dfea0001d636d2a2a43 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:48.471160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6088d4-51fb-4125-9d3b-422d6983cb6b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:48.500794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:48.521710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 142/290] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:38:48.528562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec796e-1064-4e0c-a842-a315013da6f0 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:48.581610 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:48.654960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec796e-1064-4e0c-a842-a315013da6f0 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:48.671090 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6088d4-51fb-4125-9d3b-422d6983cb6b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:48.692350 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e609f-259a-4b65-bb79-e648146fac40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:48.693441 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e609f-259a-4b65-bb79-e648146fac40 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:48.693911 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e609f-259a-4b65-bb79-e648146fac40 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:48.694511 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e609f-259a-4b65-bb79-e648146fac40 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:48.695995 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e609f-259a-4b65-bb79-e648146fac40 None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:48.697861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e609f-259a-4b65-bb79-e648146fac40 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:48.699741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 143/291] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:48 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:48.706115 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e609f-259a-4b65-bb79-e648146fac40 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:48.739948 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd800b-ed60-4197-8c1b-679c7b3e1ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:48.740947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd800b-ed60-4197-8c1b-679c7b3e1ada None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:48.741430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd800b-ed60-4197-8c1b-679c7b3e1ada None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:48.741923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd800b-ed60-4197-8c1b-679c7b3e1ada None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:48.744681 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd800b-ed60-4197-8c1b-679c7b3e1ada None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:48.746472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd800b-ed60-4197-8c1b-679c7b3e1ada None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:48.753405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 149/292] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:48 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:48.759322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd800b-ed60-4197-8c1b-679c7b3e1ada None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:48.828232 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:48.909862 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e609f-259a-4b65-bb79-e648146fac40 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:48.945971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd800b-ed60-4197-8c1b-679c7b3e1ada None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:48.951222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877eb97b-75f1-449c-a464-4a98d6096a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:48.952177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877eb97b-75f1-449c-a464-4a98d6096a82 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:48.952693 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877eb97b-75f1-449c-a464-4a98d6096a82 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:48.953168 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877eb97b-75f1-449c-a464-4a98d6096a82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:48.954106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877eb97b-75f1-449c-a464-4a98d6096a82 None admin] RBAC: Adding query filter params (role.id=0a6592b9a0b64df6938aba0a010c2256, scope.project.id=17924018ca584f3696136b838ab2fdc8, user.id=7c6eec8c701842aaa94857500668d3db) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:48.954638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877eb97b-75f1-449c-a464-4a98d6096a82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:48.956388 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877eb97b-75f1-449c-a464-4a98d6096a82 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:48.963878 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 144/293] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:48 2022] GET /identity/v3/role_assignments?user.id=7c6eec8c701842aaa94857500668d3db&scope.project.id=17924018ca584f3696136b838ab2fdc8&role.id=0a6592b9a0b64df6938aba0a010c2256 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:48.976252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:48.997424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b12b4df-50ef-43b1-8e02-037de38a5055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:48.998101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12b4df-50ef-43b1-8e02-037de38a5055 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:48.998382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12b4df-50ef-43b1-8e02-037de38a5055 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:48.998659 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b12b4df-50ef-43b1-8e02-037de38a5055 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8/users/eb9ecb752c264dfea0001d636d2a2a43/roles/2235d4074114441ea4661c3fb936dcc3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:49.000708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12b4df-50ef-43b1-8e02-037de38a5055 None admin] RBAC: Authorizing `identity:create_grant(project_id=17924018ca584f3696136b838ab2fdc8, user_id=eb9ecb752c264dfea0001d636d2a2a43, role_id=2235d4074114441ea4661c3fb936dcc3)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:49.002193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b12b4df-50ef-43b1-8e02-037de38a5055 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:49.020925 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 150/294] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:38:48 2022] PUT /identity/v3/projects/17924018ca584f3696136b838ab2fdc8/users/eb9ecb752c264dfea0001d636d2a2a43/roles/2235d4074114441ea4661c3fb936dcc3 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:38:50.301383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058a134-7c56-4a3d-ade0-75af365db81c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:50.302400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058a134-7c56-4a3d-ade0-75af365db81c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:50.302893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058a134-7c56-4a3d-ade0-75af365db81c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:50.304819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 145/295] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:50.310924 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e4442-5d11-473a-9e29-3fea3094ec4d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:50.311906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e4442-5d11-473a-9e29-3fea3094ec4d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:50.312404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e4442-5d11-473a-9e29-3fea3094ec4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:50.332668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-cf1e4442-5d11-473a-9e29-3fea3094ec4d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:50.334205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cf1e4442-5d11-473a-9e29-3fea3094ec4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:50.406867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d146a-7c6f-4175-8a35-6d7e20bd683c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:50.407495 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d146a-7c6f-4175-8a35-6d7e20bd683c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:50.408010 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d146a-7c6f-4175-8a35-6d7e20bd683c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:50.408499 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 151/296] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:50.410081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 146/297] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:50.417194 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a670a1-fb6a-4192-ba35-2b94a649610c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:50.417754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a670a1-fb6a-4192-ba35-2b94a649610c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:50.418228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a670a1-fb6a-4192-ba35-2b94a649610c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:50.437580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e2a670a1-fb6a-4192-ba35-2b94a649610c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:50.439088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e2a670a1-fb6a-4192-ba35-2b94a649610c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:50.511161 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 152/298] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:50.682041 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634c22b-77ab-4bc2-b649-ada9b50b7d6d None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:50.682339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634c22b-77ab-4bc2-b649-ada9b50b7d6d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:50.682522 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634c22b-77ab-4bc2-b649-ada9b50b7d6d None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:50.684126 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 147/299] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:50.693151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fbf1b-f246-43ce-8bcc-9bcd29751147 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:50.693445 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fbf1b-f246-43ce-8bcc-9bcd29751147 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:50.693723 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fbf1b-f246-43ce-8bcc-9bcd29751147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:50.696533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634c22b-77ab-4bc2-b649-ada9b50b7d6d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:50.721726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0c5fbf1b-f246-43ce-8bcc-9bcd29751147 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:50.723853 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0c5fbf1b-f246-43ce-8bcc-9bcd29751147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:50.757552 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:50.801454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 153/300] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:50.802559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fbf1b-f246-43ce-8bcc-9bcd29751147 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:50.836110 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634c22b-77ab-4bc2-b649-ada9b50b7d6d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:50.855460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:50.875934 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f60a70-6c95-48b0-adb2-fd6ad725ee60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:50.877067 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f60a70-6c95-48b0-adb2-fd6ad725ee60 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:50.877599 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f60a70-6c95-48b0-adb2-fd6ad725ee60 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:50.878072 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f60a70-6c95-48b0-adb2-fd6ad725ee60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:50.879495 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f60a70-6c95-48b0-adb2-fd6ad725ee60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:50.881453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f60a70-6c95-48b0-adb2-fd6ad725ee60 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:50.883356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 148/301] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:50.889824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f60a70-6c95-48b0-adb2-fd6ad725ee60 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:50.917915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fbf1b-f246-43ce-8bcc-9bcd29751147 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:50.955739 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda3aeb-253c-49b4-804c-6cd1e14b502a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:50.956353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda3aeb-253c-49b4-804c-6cd1e14b502a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:50.956632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda3aeb-253c-49b4-804c-6cd1e14b502a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:50.956969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fda3aeb-253c-49b4-804c-6cd1e14b502a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:50.968930 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fda3aeb-253c-49b4-804c-6cd1e14b502a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-9fda3aeb-253c-49b4-804c-6cd1e14b502a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:50.974334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:50.975248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:50.976721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:50.976721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:50.976721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:50.976721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:50.976721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:50.976721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:50.976721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:50.976721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:50.976721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:50.976721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:50.976721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:50.976721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:50.976721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 154/302] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:50.981356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda3aeb-253c-49b4-804c-6cd1e14b502a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:51.004364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f60a70-6c95-48b0-adb2-fd6ad725ee60 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:51.012159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:51.042196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699531fb-d720-41da-a211-18e38f6d39df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:51.043202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699531fb-d720-41da-a211-18e38f6d39df None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:51.043723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699531fb-d720-41da-a211-18e38f6d39df None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:51.044227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699531fb-d720-41da-a211-18e38f6d39df None admin] PATH_INFO: `/v3/groups` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:51.045054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699531fb-d720-41da-a211-18e38f6d39df None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:51.046952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699531fb-d720-41da-a211-18e38f6d39df None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:51.076136 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 149/303] 10.222.0.60 () {60 vars in 1134 bytes} [Mon Sep 19 14:38:50 2022] POST /identity/v3/groups => generated 229 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:38:51.081008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699531fb-d720-41da-a211-18e38f6d39df None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:51.097718 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fda3aeb-253c-49b4-804c-6cd1e14b502a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:51.139094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7a439-8614-4e6d-b5c7-c03054e5d3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:51.139742 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7a439-8614-4e6d-b5c7-c03054e5d3d6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:51.140042 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7a439-8614-4e6d-b5c7-c03054e5d3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:51.140313 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7a439-8614-4e6d-b5c7-c03054e5d3d6 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:51.140658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:51.142329 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7a439-8614-4e6d-b5c7-c03054e5d3d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:51.142668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7a439-8614-4e6d-b5c7-c03054e5d3d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:51.144306 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7a439-8614-4e6d-b5c7-c03054e5d3d6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:51.161340 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 155/304] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:51.162558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7a439-8614-4e6d-b5c7-c03054e5d3d6 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:51.213130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699531fb-d720-41da-a211-18e38f6d39df None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:51.253634 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309f676-5469-488b-9a02-ecb889b49a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:51.254729 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309f676-5469-488b-9a02-ecb889b49a23 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:51.255162 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309f676-5469-488b-9a02-ecb889b49a23 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:51.255582 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309f676-5469-488b-9a02-ecb889b49a23 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:51.265230 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:51.270941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309f676-5469-488b-9a02-ecb889b49a23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-2309f676-5469-488b-9a02-ecb889b49a23 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:51.277423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:51.279102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:51.279967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:51.279967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:51.279967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:51.279967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:51.279967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:51.279967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:51.279967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:51.279967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:51.279967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:51.279967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:51.279967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:51.279967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:51.282519 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 150/305] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:51.283333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7a439-8614-4e6d-b5c7-c03054e5d3d6 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:51.288755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309f676-5469-488b-9a02-ecb889b49a23 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:51.328303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d44077-280f-4911-ab22-55963566f63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:51.329727 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d44077-280f-4911-ab22-55963566f63c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:51.330190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d44077-280f-4911-ab22-55963566f63c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:51.330637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d44077-280f-4911-ab22-55963566f63c None admin] PATH_INFO: `/v3/users/7854aa333c774071aa916b8c294be9bf` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:51.348303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d44077-280f-4911-ab22-55963566f63c None admin] RBAC: Authorizing `identity:get_user(user_id=7854aa333c774071aa916b8c294be9bf)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:51.350815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d44077-280f-4911-ab22-55963566f63c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:51.352737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 156/306] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:51 2022] GET /identity/v3/users/7854aa333c774071aa916b8c294be9bf => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:51.359815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d44077-280f-4911-ab22-55963566f63c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:51.408348 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309f676-5469-488b-9a02-ecb889b49a23 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:51.411846 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:51.447364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1d3c4-c6a2-454a-9795-b8167322152e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:51.448027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1d3c4-c6a2-454a-9795-b8167322152e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:51.448347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1d3c4-c6a2-454a-9795-b8167322152e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:51.448654 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1d3c4-c6a2-454a-9795-b8167322152e None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:51.449414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1d3c4-c6a2-454a-9795-b8167322152e None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:51.449762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1d3c4-c6a2-454a-9795-b8167322152e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:51.452169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1d3c4-c6a2-454a-9795-b8167322152e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:51.461710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 151/307] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:51.462915 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c3952-bd06-4b20-b86d-f4dfff2b39b8 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:51.463263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c3952-bd06-4b20-b86d-f4dfff2b39b8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:51.463484 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c3952-bd06-4b20-b86d-f4dfff2b39b8 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:51.468869 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 152/308] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:51.468869 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c3952-bd06-4b20-b86d-f4dfff2b39b8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:51.480652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d44077-280f-4911-ab22-55963566f63c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:51.519534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c8cf7-9ed2-4fa7-99b6-6361285027e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:51.520264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c8cf7-9ed2-4fa7-99b6-6361285027e3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:51.520546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c8cf7-9ed2-4fa7-99b6-6361285027e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:51.520874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c8cf7-9ed2-4fa7-99b6-6361285027e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:51.521609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c8cf7-9ed2-4fa7-99b6-6361285027e3 None admin] RBAC: Adding query filter params (role.id=2235d4074114441ea4661c3fb936dcc3, user.id=7854aa333c774071aa916b8c294be9bf, scope.system=all) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:51.521912 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c8cf7-9ed2-4fa7-99b6-6361285027e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:51.524208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7c8cf7-9ed2-4fa7-99b6-6361285027e3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:51.530223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 157/309] 10.222.0.60 () {56 vars in 1308 bytes} [Mon Sep 19 14:38:51 2022] GET /identity/v3/role_assignments?user.id=7854aa333c774071aa916b8c294be9bf&scope.system=all&role.id=2235d4074114441ea4661c3fb936dcc3 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:51.531295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdb86e-02fe-4f98-b820-9739181d97c1 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:51.531575 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdb86e-02fe-4f98-b820-9739181d97c1 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:51.531895 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bdb86e-02fe-4f98-b820-9739181d97c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:51.554720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f1bdb86e-02fe-4f98-b820-9739181d97c1 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:51.556005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f1bdb86e-02fe-4f98-b820-9739181d97c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:51.561718 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:51.630669 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c3952-bd06-4b20-b86d-f4dfff2b39b8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:51.660368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 158/310] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:51.681615 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa4cd5-2c83-48a8-af82-fddd9c645f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:51.682336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa4cd5-2c83-48a8-af82-fddd9c645f02 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:51.682688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa4cd5-2c83-48a8-af82-fddd9c645f02 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:51.682959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa4cd5-2c83-48a8-af82-fddd9c645f02 None admin] PATH_INFO: `/v3/users/4f39adfbcf654dd6a0900eb89af6fab8` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:51.684235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa4cd5-2c83-48a8-af82-fddd9c645f02 None admin] RBAC: Authorizing `identity:get_user(user_id=4f39adfbcf654dd6a0900eb89af6fab8)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:51.686296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aa4cd5-2c83-48a8-af82-fddd9c645f02 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:51.688447 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 153/311] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:51 2022] GET /identity/v3/users/4f39adfbcf654dd6a0900eb89af6fab8 => generated 293 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:51.694057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdb86e-02fe-4f98-b820-9739181d97c1 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:51.801813 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bdb86e-02fe-4f98-b820-9739181d97c1 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:51.837625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8caf6-069b-400f-8909-b4dba31bc1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:51.838345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8caf6-069b-400f-8909-b4dba31bc1ef None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:51.838628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8caf6-069b-400f-8909-b4dba31bc1ef None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:51.838906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8caf6-069b-400f-8909-b4dba31bc1ef None admin] PATH_INFO: `/v3/system/users/4f39adfbcf654dd6a0900eb89af6fab8/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:51.840404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8caf6-069b-400f-8909-b4dba31bc1ef None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4f39adfbcf654dd6a0900eb89af6fab8, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:51.841850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc8caf6-069b-400f-8909-b4dba31bc1ef None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:51.851992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 159/312] 10.222.0.60 () {58 vars in 1334 bytes} [Mon Sep 19 14:38:51 2022] PUT /identity/v3/system/users/4f39adfbcf654dd6a0900eb89af6fab8/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:38:51.977707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa4cd5-2c83-48a8-af82-fddd9c645f02 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:52.016423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:52.115009 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa4cd5-2c83-48a8-af82-fddd9c645f02 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:52.157280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990991d3-574b-4b68-9343-590735bbb116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:52.158308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990991d3-574b-4b68-9343-590735bbb116 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:52.158824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990991d3-574b-4b68-9343-590735bbb116 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:52.159485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990991d3-574b-4b68-9343-590735bbb116 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:52.173257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990991d3-574b-4b68-9343-590735bbb116 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-990991d3-574b-4b68-9343-590735bbb116 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:52.179894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:52.180789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:52.181616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:52.181616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:52.181616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:52.181616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:52.181616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:52.181616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:52.181616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:52.181616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:52.181616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:52.181616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:52.181616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:52.181616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:52.184192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 154/313] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:52.190762 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8caf6-069b-400f-8909-b4dba31bc1ef None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:52.197814 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:52.319812 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8caf6-069b-400f-8909-b4dba31bc1ef None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:52.362972 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9a7ef-1dcf-48f6-aec1-523eb5d83f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:52.364414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9a7ef-1dcf-48f6-aec1-523eb5d83f31 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:52.364913 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9a7ef-1dcf-48f6-aec1-523eb5d83f31 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:52.365399 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9a7ef-1dcf-48f6-aec1-523eb5d83f31 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:52.366258 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9a7ef-1dcf-48f6-aec1-523eb5d83f31 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:52.366758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9a7ef-1dcf-48f6-aec1-523eb5d83f31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:52.368397 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9a7ef-1dcf-48f6-aec1-523eb5d83f31 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:52.382046 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 160/314] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:52.388338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990991d3-574b-4b68-9343-590735bbb116 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:52.514343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:52.523147 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990991d3-574b-4b68-9343-590735bbb116 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:52.569591 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b72f3-8c3f-4139-bc2b-b91e0eacc142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:52.570224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b72f3-8c3f-4139-bc2b-b91e0eacc142 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:52.570501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b72f3-8c3f-4139-bc2b-b91e0eacc142 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:52.570769 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b72f3-8c3f-4139-bc2b-b91e0eacc142 None admin] PATH_INFO: `/v3/users/eb9ecb752c264dfea0001d636d2a2a43` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:52.571913 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b72f3-8c3f-4139-bc2b-b91e0eacc142 None admin] RBAC: Authorizing `identity:get_user(user_id=eb9ecb752c264dfea0001d636d2a2a43)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:52.573359 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b72f3-8c3f-4139-bc2b-b91e0eacc142 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:52.575285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 155/315] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:52 2022] GET /identity/v3/users/eb9ecb752c264dfea0001d636d2a2a43 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:52.580893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9a7ef-1dcf-48f6-aec1-523eb5d83f31 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:52.660945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:52.699061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9a7ef-1dcf-48f6-aec1-523eb5d83f31 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:52.734729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8b7ef-556f-45aa-94eb-f0f003e692ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:52.735392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8b7ef-556f-45aa-94eb-f0f003e692ce None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:52.735676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8b7ef-556f-45aa-94eb-f0f003e692ce None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:52.735984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8b7ef-556f-45aa-94eb-f0f003e692ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:52.806849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8b7ef-556f-45aa-94eb-f0f003e692ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:52.808346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8b7ef-556f-45aa-94eb-f0f003e692ce None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:52.873996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:52.916908 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 161/316] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:38:52.923678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b72f3-8c3f-4139-bc2b-b91e0eacc142 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:53.037722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b72f3-8c3f-4139-bc2b-b91e0eacc142 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:53.076603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101b87b-7cf7-4b35-9c2f-faec92b28208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:53.077430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101b87b-7cf7-4b35-9c2f-faec92b28208 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:53.077711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101b87b-7cf7-4b35-9c2f-faec92b28208 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:53.077982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101b87b-7cf7-4b35-9c2f-faec92b28208 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:53.079123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101b87b-7cf7-4b35-9c2f-faec92b28208 None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:53.080610 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101b87b-7cf7-4b35-9c2f-faec92b28208 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:53.082536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 156/317] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:52 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:53.093505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8b7ef-556f-45aa-94eb-f0f003e692ce None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:53.203413 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8b7ef-556f-45aa-94eb-f0f003e692ce None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:53.238807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1786c3f4-9d30-4a65-a0f2-209a9aa1ecde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:53.239415 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786c3f4-9d30-4a65-a0f2-209a9aa1ecde None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:53.239696 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786c3f4-9d30-4a65-a0f2-209a9aa1ecde None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:53.240000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1786c3f4-9d30-4a65-a0f2-209a9aa1ecde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:53.240725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786c3f4-9d30-4a65-a0f2-209a9aa1ecde None admin] RBAC: Adding query filter params (role.id=2235d4074114441ea4661c3fb936dcc3, scope.project.id=17924018ca584f3696136b838ab2fdc8, user.id=eb9ecb752c264dfea0001d636d2a2a43) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:53.241031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786c3f4-9d30-4a65-a0f2-209a9aa1ecde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:53.242462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1786c3f4-9d30-4a65-a0f2-209a9aa1ecde None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:53.248893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 162/318] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:38:53 2022] GET /identity/v3/role_assignments?user.id=eb9ecb752c264dfea0001d636d2a2a43&scope.project.id=17924018ca584f3696136b838ab2fdc8&role.id=2235d4074114441ea4661c3fb936dcc3 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:53.553671 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05f892-b4b3-4aac-82a3-5eda0a611e61 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:53.554154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05f892-b4b3-4aac-82a3-5eda0a611e61 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:53.554154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05f892-b4b3-4aac-82a3-5eda0a611e61 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:53.555504 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 157/319] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:53.561124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686202aa-24e3-466e-85fb-35c1cc56cb93 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:53.561510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686202aa-24e3-466e-85fb-35c1cc56cb93 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:53.561685 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686202aa-24e3-466e-85fb-35c1cc56cb93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:53.579979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-686202aa-24e3-466e-85fb-35c1cc56cb93 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:53.581154 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-686202aa-24e3-466e-85fb-35c1cc56cb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:53.605926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:53.652483 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 163/320] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:53.811127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c8d29-2faa-40e4-9909-061e44123ef2 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:53.811410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c8d29-2faa-40e4-9909-061e44123ef2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:53.811605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c8d29-2faa-40e4-9909-061e44123ef2 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:53.813121 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 158/321] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:53.818411 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d310b6-3711-484b-b266-c472e9847b39 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:53.818796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d310b6-3711-484b-b266-c472e9847b39 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:53.819080 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d310b6-3711-484b-b266-c472e9847b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:53.839062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-24d310b6-3711-484b-b266-c472e9847b39 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:53.840294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-24d310b6-3711-484b-b266-c472e9847b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:53.914796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 164/322] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:53.930079 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c8d29-2faa-40e4-9909-061e44123ef2 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:53.961267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:54.044036 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c8d29-2faa-40e4-9909-061e44123ef2 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:54.080407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4596f-177e-4cfb-a95c-25c0c9c80f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:54.080970 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4596f-177e-4cfb-a95c-25c0c9c80f7b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:54.081255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4596f-177e-4cfb-a95c-25c0c9c80f7b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:54.081614 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4596f-177e-4cfb-a95c-25c0c9c80f7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:54.082751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4596f-177e-4cfb-a95c-25c0c9c80f7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:54.085745 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4596f-177e-4cfb-a95c-25c0c9c80f7b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:54.087863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 159/323] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:38:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:54.092286 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d310b6-3711-484b-b266-c472e9847b39 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:54.191696 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4596f-177e-4cfb-a95c-25c0c9c80f7b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:54.193893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:54.206922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d310b6-3711-484b-b266-c472e9847b39 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:54.243591 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8fb85-a494-46f8-baba-182b440ef4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:54.244233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8fb85-a494-46f8-baba-182b440ef4d2 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:54.244523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8fb85-a494-46f8-baba-182b440ef4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:54.244789 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8fb85-a494-46f8-baba-182b440ef4d2 None admin] PATH_INFO: `/v3/groups` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:54.245415 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8fb85-a494-46f8-baba-182b440ef4d2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:54.246890 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8fb85-a494-46f8-baba-182b440ef4d2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:54.263277 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 165/324] 10.222.0.60 () {60 vars in 1134 bytes} [Mon Sep 19 14:38:54 2022] POST /identity/v3/groups => generated 226 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:38:54.264595 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3709078-2731-4e20-bf21-287951897f6b None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:54.264886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3709078-2731-4e20-bf21-287951897f6b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:54.265160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3709078-2731-4e20-bf21-287951897f6b None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:54.266430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 166/325] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:54.272523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc18cd-bebb-42a1-abcc-a723ba1d3aa0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:54.272813 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc18cd-bebb-42a1-abcc-a723ba1d3aa0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:54.273093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc18cd-bebb-42a1-abcc-a723ba1d3aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:54.296074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-03dc18cd-bebb-42a1-abcc-a723ba1d3aa0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:54.297902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-03dc18cd-bebb-42a1-abcc-a723ba1d3aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:54.308952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:54.313513 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4596f-177e-4cfb-a95c-25c0c9c80f7b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:54.357315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed65f6-9327-4b19-9062-50172d6acb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:54.358199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed65f6-9327-4b19-9062-50172d6acb55 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:54.358479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed65f6-9327-4b19-9062-50172d6acb55 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:54.359280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed65f6-9327-4b19-9062-50172d6acb55 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:54.373289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed65f6-9327-4b19-9062-50172d6acb55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-02ed65f6-9327-4b19-9062-50172d6acb55 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:54.378324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:54.379287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:54.380379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:54.380379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:54.380379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:54.380379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:54.380379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:54.380379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:54.380379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:54.380379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:54.380379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:54.380379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:54.380379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:54.380379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:54.380379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 160/326] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:54.385372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 167/327] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:38:54.392241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed65f6-9327-4b19-9062-50172d6acb55 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:54.495872 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed65f6-9327-4b19-9062-50172d6acb55 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:54.531188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e623ef2-b162-457d-adbe-79573d3e8b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:54.531801 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e623ef2-b162-457d-adbe-79573d3e8b20 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:54.532139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e623ef2-b162-457d-adbe-79573d3e8b20 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:54.532417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e623ef2-b162-457d-adbe-79573d3e8b20 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:54.533076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e623ef2-b162-457d-adbe-79573d3e8b20 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:54.533373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e623ef2-b162-457d-adbe-79573d3e8b20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:54.534762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e623ef2-b162-457d-adbe-79573d3e8b20 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:54.544391 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 161/328] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:38:54.550252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc18cd-bebb-42a1-abcc-a723ba1d3aa0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:54.639183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:54.662818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc18cd-bebb-42a1-abcc-a723ba1d3aa0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:54.668798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e623ef2-b162-457d-adbe-79573d3e8b20 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:54.694417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:54.710269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a94ef-4cb1-4f59-9831-3b29cb70ca08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:54.710903 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a94ef-4cb1-4f59-9831-3b29cb70ca08 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:54.711185 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a94ef-4cb1-4f59-9831-3b29cb70ca08 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:54.711450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a94ef-4cb1-4f59-9831-3b29cb70ca08 None admin] PATH_INFO: `/v3/users/7854aa333c774071aa916b8c294be9bf` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:54.712606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a94ef-4cb1-4f59-9831-3b29cb70ca08 None admin] RBAC: Authorizing `identity:get_user(user_id=7854aa333c774071aa916b8c294be9bf)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:54.714037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a94ef-4cb1-4f59-9831-3b29cb70ca08 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:54.715908 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 168/329] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:54 2022] GET /identity/v3/users/7854aa333c774071aa916b8c294be9bf => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:54.722998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a94ef-4cb1-4f59-9831-3b29cb70ca08 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:54.796661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e623ef2-b162-457d-adbe-79573d3e8b20 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:54.841726 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675369b4-9baa-4062-8616-cc88a29e4741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:54.842380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675369b4-9baa-4062-8616-cc88a29e4741 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:54.842666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675369b4-9baa-4062-8616-cc88a29e4741 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:54.842944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675369b4-9baa-4062-8616-cc88a29e4741 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:54.858260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a94ef-4cb1-4f59-9831-3b29cb70ca08 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:54.859162 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675369b4-9baa-4062-8616-cc88a29e4741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-675369b4-9baa-4062-8616-cc88a29e4741 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:54.866497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:54.867648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:54.869473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:54.869473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:54.869473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:54.869473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:54.869473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:54.869473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:54.869473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:54.869473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:54.869473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:54.869473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:54.869473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:54.869473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:54.869473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 162/330] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:38:54.875021 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675369b4-9baa-4062-8616-cc88a29e4741 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:54.913388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b770e02-18ed-4372-99fe-75cc625c3c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:54.914304 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b770e02-18ed-4372-99fe-75cc625c3c09 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:54.914730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b770e02-18ed-4372-99fe-75cc625c3c09 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:54.915145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b770e02-18ed-4372-99fe-75cc625c3c09 None admin] PATH_INFO: `/v3/system/users/7854aa333c774071aa916b8c294be9bf/roles/2235d4074114441ea4661c3fb936dcc3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:54.916801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b770e02-18ed-4372-99fe-75cc625c3c09 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7854aa333c774071aa916b8c294be9bf, role_id=2235d4074114441ea4661c3fb936dcc3)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:54.918341 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b770e02-18ed-4372-99fe-75cc625c3c09 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:54.929843 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 169/331] 10.222.0.60 () {58 vars in 1334 bytes} [Mon Sep 19 14:38:54 2022] PUT /identity/v3/system/users/7854aa333c774071aa916b8c294be9bf/roles/2235d4074114441ea4661c3fb936dcc3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:38:54.988302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:55.010537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675369b4-9baa-4062-8616-cc88a29e4741 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:55.056481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e46fff-aafd-43e5-bcc3-13721bb1d4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:55.057120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e46fff-aafd-43e5-bcc3-13721bb1d4f4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:55.057406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e46fff-aafd-43e5-bcc3-13721bb1d4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:55.057679 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e46fff-aafd-43e5-bcc3-13721bb1d4f4 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:55.058424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e46fff-aafd-43e5-bcc3-13721bb1d4f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:55.058731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e46fff-aafd-43e5-bcc3-13721bb1d4f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:55.060725 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e46fff-aafd-43e5-bcc3-13721bb1d4f4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:55.081396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 163/332] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:55.087064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b770e02-18ed-4372-99fe-75cc625c3c09 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:55.112084 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:55.219051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b770e02-18ed-4372-99fe-75cc625c3c09 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:55.260048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a547a-e32b-4442-84e0-63fb93b6f64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:55.260696 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a547a-e32b-4442-84e0-63fb93b6f64b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:55.260974 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a547a-e32b-4442-84e0-63fb93b6f64b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:55.261250 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a547a-e32b-4442-84e0-63fb93b6f64b None admin] PATH_INFO: `/v3/users/4f39adfbcf654dd6a0900eb89af6fab8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:55.262404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a547a-e32b-4442-84e0-63fb93b6f64b None admin] RBAC: Authorizing `identity:get_user(user_id=4f39adfbcf654dd6a0900eb89af6fab8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:55.264434 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a547a-e32b-4442-84e0-63fb93b6f64b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:55.266336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 170/333] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:55 2022] GET /identity/v3/users/4f39adfbcf654dd6a0900eb89af6fab8 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:55.272091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e46fff-aafd-43e5-bcc3-13721bb1d4f4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:55.420348 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e46fff-aafd-43e5-bcc3-13721bb1d4f4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:55.474246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f9ee1-f3ce-452a-81a3-e285f0f51f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:55.474785 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f9ee1-f3ce-452a-81a3-e285f0f51f93 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:55.475065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f9ee1-f3ce-452a-81a3-e285f0f51f93 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:55.475336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f9ee1-f3ce-452a-81a3-e285f0f51f93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:55.476088 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f9ee1-f3ce-452a-81a3-e285f0f51f93 None admin] RBAC: Adding query filter params (role.id=0a6592b9a0b64df6938aba0a010c2256, user.id=4f39adfbcf654dd6a0900eb89af6fab8, scope.system=all) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:55.476396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f9ee1-f3ce-452a-81a3-e285f0f51f93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:55.477850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f9ee1-f3ce-452a-81a3-e285f0f51f93 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:55.495117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 164/334] 10.222.0.60 () {56 vars in 1308 bytes} [Mon Sep 19 14:38:55 2022] GET /identity/v3/role_assignments?user.id=4f39adfbcf654dd6a0900eb89af6fab8&scope.system=all&role.id=0a6592b9a0b64df6938aba0a010c2256 => generated 510 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:55.495406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:57.242460 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b030fb-16c4-4c16-a2e9-65c630a932d1 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:57.243430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b030fb-16c4-4c16-a2e9-65c630a932d1 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:57.243945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b030fb-16c4-4c16-a2e9-65c630a932d1 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:57.245549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 171/335] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:57.258833 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39b57d-274f-4b90-916e-1f3f44d11e4a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:57.259362 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39b57d-274f-4b90-916e-1f3f44d11e4a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:57.259906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39b57d-274f-4b90-916e-1f3f44d11e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:57.288879 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2f39b57d-274f-4b90-916e-1f3f44d11e4a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:57.290097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2f39b57d-274f-4b90-916e-1f3f44d11e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:57.379081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 165/336] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:57.557933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757030ea-cc6f-4783-91c7-2b4988c50933 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:57.558632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757030ea-cc6f-4783-91c7-2b4988c50933 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:57.559115 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757030ea-cc6f-4783-91c7-2b4988c50933 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:57.561323 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 172/337] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:57.567246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abb1f2-015d-4c6a-a659-391ecd7dc624 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:57.567539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abb1f2-015d-4c6a-a659-391ecd7dc624 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:57.567840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2abb1f2-015d-4c6a-a659-391ecd7dc624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:57.597131 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b2abb1f2-015d-4c6a-a659-391ecd7dc624 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:57.598391 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b2abb1f2-015d-4c6a-a659-391ecd7dc624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:57.701605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 166/338] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:57.783077 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e2bf5-b3a2-4d88-aac5-cf732500dd82 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:57.783980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e2bf5-b3a2-4d88-aac5-cf732500dd82 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:57.784762 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e2bf5-b3a2-4d88-aac5-cf732500dd82 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:57.787083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 173/339] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:57.793470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af642da-af91-4f29-aeaf-932640f96c85 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:57.794277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af642da-af91-4f29-aeaf-932640f96c85 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:57.794562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af642da-af91-4f29-aeaf-932640f96c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:57.819695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-7af642da-af91-4f29-aeaf-932640f96c85 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:57.820997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7af642da-af91-4f29-aeaf-932640f96c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:57.890884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:57.904701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 167/340] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:57.913843 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e2bf5-b3a2-4d88-aac5-cf732500dd82 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:58.026491 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:58.052238 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e2bf5-b3a2-4d88-aac5-cf732500dd82 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:58.103450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd662ff1-00a3-4f71-b3aa-486b4bc0af1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:58.104135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd662ff1-00a3-4f71-b3aa-486b4bc0af1d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:58.104417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd662ff1-00a3-4f71-b3aa-486b4bc0af1d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:58.104693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd662ff1-00a3-4f71-b3aa-486b4bc0af1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:58.124493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd662ff1-00a3-4f71-b3aa-486b4bc0af1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-fd662ff1-00a3-4f71-b3aa-486b4bc0af1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:58.129428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:58.130369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:58.131198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:58.131198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:58.131198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:58.131198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:58.131198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:58.131198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:58.131198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:58.131198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:58.131198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:58.131198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:58.131198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:58.131198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:58.131939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 174/341] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:58.136316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af642da-af91-4f29-aeaf-932640f96c85 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:58.249993 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af642da-af91-4f29-aeaf-932640f96c85 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:58.289925 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4e207-2d8c-42c8-8c82-40d18111fc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:58.290542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4e207-2d8c-42c8-8c82-40d18111fc4c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:58.290775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4e207-2d8c-42c8-8c82-40d18111fc4c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:58.291007 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4e207-2d8c-42c8-8c82-40d18111fc4c None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:58.291687 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4e207-2d8c-42c8-8c82-40d18111fc4c None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:58.292026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4e207-2d8c-42c8-8c82-40d18111fc4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:58.293412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4e207-2d8c-42c8-8c82-40d18111fc4c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:58.307887 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd662ff1-00a3-4f71-b3aa-486b4bc0af1d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:58.311399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 168/342] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:58.316650 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4e207-2d8c-42c8-8c82-40d18111fc4c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:58.443301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:58.458310 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd662ff1-00a3-4f71-b3aa-486b4bc0af1d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:58.530610 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e960e-295b-4f63-bcc1-94f498596059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:58.531256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e960e-295b-4f63-bcc1-94f498596059 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:58.531537 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e960e-295b-4f63-bcc1-94f498596059 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:58.531808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e960e-295b-4f63-bcc1-94f498596059 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:58.553058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e960e-295b-4f63-bcc1-94f498596059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-1f8e960e-295b-4f63-bcc1-94f498596059 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:58.558228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:58.559116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:58.560093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:58.560093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:58.560093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:58.560093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:58.560093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:58.560093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:58.560093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:58.560093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:58.560093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:58.560093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:58.560093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 19 14:38:58.560093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:58.560093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:58.562836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 175/343] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:38:58.563714 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4e207-2d8c-42c8-8c82-40d18111fc4c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:58.563858 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e960e-295b-4f63-bcc1-94f498596059 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:58.626221 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc9526-022a-4080-b989-1a62cc20da17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:58.627303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc9526-022a-4080-b989-1a62cc20da17 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:58.627792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc9526-022a-4080-b989-1a62cc20da17 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:58.628320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc9526-022a-4080-b989-1a62cc20da17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:58.714567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e960e-295b-4f63-bcc1-94f498596059 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:58.728666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc9526-022a-4080-b989-1a62cc20da17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:58.730259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc9526-022a-4080-b989-1a62cc20da17 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:58.760493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89319179-6be4-4e56-ba62-6fb290737b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:58.761496 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89319179-6be4-4e56-ba62-6fb290737b62 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:58.763193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89319179-6be4-4e56-ba62-6fb290737b62 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:58.763690 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89319179-6be4-4e56-ba62-6fb290737b62 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:58.780202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89319179-6be4-4e56-ba62-6fb290737b62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-89319179-6be4-4e56-ba62-6fb290737b62 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:58.786744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:58.787712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:58.792003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:58.792003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:58.792003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:58.792003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:58.792003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:58.792003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:58.792003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:58.792003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:58.792003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:58.792003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:58.792003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:38:58.792003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:38:58.792003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 176/344] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:38:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:58.792003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89319179-6be4-4e56-ba62-6fb290737b62 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:58.813172 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:58.879135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 169/345] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 14:38:58.882262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f4a18-d86f-42d1-bd9e-dd1b7cb20a0d None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:58.882262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f4a18-d86f-42d1-bd9e-dd1b7cb20a0d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:58.882262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266f4a18-d86f-42d1-bd9e-dd1b7cb20a0d None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:58.883094 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 170/346] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:58.883884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f4a18-d86f-42d1-bd9e-dd1b7cb20a0d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:58.893341 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:58.946025 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89319179-6be4-4e56-ba62-6fb290737b62 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:58.987587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c542f0-cee9-4b18-9d14-2db8912993e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:58.989283 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c542f0-cee9-4b18-9d14-2db8912993e3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:58.989810 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c542f0-cee9-4b18-9d14-2db8912993e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:58.990629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c542f0-cee9-4b18-9d14-2db8912993e3 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:58.991810 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c542f0-cee9-4b18-9d14-2db8912993e3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:58.992405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c542f0-cee9-4b18-9d14-2db8912993e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:58.994031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c542f0-cee9-4b18-9d14-2db8912993e3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:59.005400 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 177/347] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:38:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:59.007184 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320cf58-0ed7-4b82-b35c-2d39359ce3c4 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:59.007933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320cf58-0ed7-4b82-b35c-2d39359ce3c4 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:59.008607 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320cf58-0ed7-4b82-b35c-2d39359ce3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:59.017421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266f4a18-d86f-42d1-bd9e-dd1b7cb20a0d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:59.033855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9320cf58-0ed7-4b82-b35c-2d39359ce3c4 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:59.035401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9320cf58-0ed7-4b82-b35c-2d39359ce3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:59.057896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640ea13-baca-44ee-8e66-4ef3bee6e8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:59.058509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640ea13-baca-44ee-8e66-4ef3bee6e8c6 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:59.058790 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640ea13-baca-44ee-8e66-4ef3bee6e8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:59.059059 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640ea13-baca-44ee-8e66-4ef3bee6e8c6 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:59.059765 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640ea13-baca-44ee-8e66-4ef3bee6e8c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:59.060119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640ea13-baca-44ee-8e66-4ef3bee6e8c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:59.062119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640ea13-baca-44ee-8e66-4ef3bee6e8c6 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:59.076103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 171/348] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:38:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:59.078251 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640ea13-baca-44ee-8e66-4ef3bee6e8c6 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:59.104850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:59.125485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 178/349] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:59.126786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320cf58-0ed7-4b82-b35c-2d39359ce3c4 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:59.197649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:59.233462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640ea13-baca-44ee-8e66-4ef3bee6e8c6 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:59.254128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320cf58-0ed7-4b82-b35c-2d39359ce3c4 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:59.281343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da00fe6-63ee-41a6-968c-96894b77e4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:59.282123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da00fe6-63ee-41a6-968c-96894b77e4eb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:59.282509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da00fe6-63ee-41a6-968c-96894b77e4eb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:59.283580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da00fe6-63ee-41a6-968c-96894b77e4eb None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:59.285271 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da00fe6-63ee-41a6-968c-96894b77e4eb None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:59.287646 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da00fe6-63ee-41a6-968c-96894b77e4eb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:59.290477 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 172/350] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:59 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:59.292492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eb5f5-99c8-43ea-8ab2-e5ee7f3020c8 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:59.292783 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eb5f5-99c8-43ea-8ab2-e5ee7f3020c8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:59.293061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2eb5f5-99c8-43ea-8ab2-e5ee7f3020c8 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:59.295918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 173/351] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:59.296771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2eb5f5-99c8-43ea-8ab2-e5ee7f3020c8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:59.309332 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296b3d7-cb43-46b7-b710-c4c2dad1fb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:59.310692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296b3d7-cb43-46b7-b710-c4c2dad1fb33 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:59.311627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296b3d7-cb43-46b7-b710-c4c2dad1fb33 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:59.311627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296b3d7-cb43-46b7-b710-c4c2dad1fb33 None admin] PATH_INFO: `/v3/users/7854aa333c774071aa916b8c294be9bf` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:59.312493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7296b3d7-cb43-46b7-b710-c4c2dad1fb33 None admin] RBAC: Authorizing `identity:get_user(user_id=7854aa333c774071aa916b8c294be9bf)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:59.313999 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7296b3d7-cb43-46b7-b710-c4c2dad1fb33 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:59.315903 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 179/352] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:38:59 2022] GET /identity/v3/users/7854aa333c774071aa916b8c294be9bf => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:59.317591 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296b3d7-cb43-46b7-b710-c4c2dad1fb33 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:59.421645 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:59.447487 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2eb5f5-99c8-43ea-8ab2-e5ee7f3020c8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:59.449109 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7296b3d7-cb43-46b7-b710-c4c2dad1fb33 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:59.478928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:59.491668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361aed3-4987-4d38-9b16-43f85cc3c504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:59.492254 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd7b65-373c-4eaf-b507-1c28a5eb7db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:59.492436 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361aed3-4987-4d38-9b16-43f85cc3c504 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:59.492896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd7b65-373c-4eaf-b507-1c28a5eb7db8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:59.493126 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361aed3-4987-4d38-9b16-43f85cc3c504 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:59.493234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd7b65-373c-4eaf-b507-1c28a5eb7db8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:59.493437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd7b65-373c-4eaf-b507-1c28a5eb7db8 None admin] PATH_INFO: `/v3/groups/2620063028584525a9ef3d5e557d96b6` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:59.493542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361aed3-4987-4d38-9b16-43f85cc3c504 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:59.503614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd7b65-373c-4eaf-b507-1c28a5eb7db8 None admin] RBAC: Authorizing `identity:get_group(group_id=2620063028584525a9ef3d5e557d96b6)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:59.503614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd7b65-373c-4eaf-b507-1c28a5eb7db8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:59.504125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 180/353] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Sep 19 14:38:59 2022] GET /identity/v3/groups/2620063028584525a9ef3d5e557d96b6 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:59.505457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c770909-4ce5-47da-a077-785bb5d53078 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:59.505747 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c770909-4ce5-47da-a077-785bb5d53078 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:59.506023 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c770909-4ce5-47da-a077-785bb5d53078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:59.527627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3c770909-4ce5-47da-a077-785bb5d53078 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:38:59.529723 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3c770909-4ce5-47da-a077-785bb5d53078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:38:59.573135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361aed3-4987-4d38-9b16-43f85cc3c504 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:59.574600 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361aed3-4987-4d38-9b16-43f85cc3c504 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:59.607612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 181/354] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:38:59.609057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c770909-4ce5-47da-a077-785bb5d53078 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:59.669155 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:59.692458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 174/355] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:38:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:38:59.694214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf6607-6b8a-4225-b0db-440c616867e8 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:59.694714 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf6607-6b8a-4225-b0db-440c616867e8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:59.695202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf6607-6b8a-4225-b0db-440c616867e8 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:59.696671 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 175/356] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:38:59.697629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf6607-6b8a-4225-b0db-440c616867e8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:59.739933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c770909-4ce5-47da-a077-785bb5d53078 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:59.755646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:59.781980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3157c35-f22b-4f0c-a2ac-bc9a2fb53a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:59.782605 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3157c35-f22b-4f0c-a2ac-bc9a2fb53a2b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:59.782872 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3157c35-f22b-4f0c-a2ac-bc9a2fb53a2b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:59.783147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3157c35-f22b-4f0c-a2ac-bc9a2fb53a2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:59.785860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3157c35-f22b-4f0c-a2ac-bc9a2fb53a2b None admin] RBAC: Adding query filter params (role.id=2235d4074114441ea4661c3fb936dcc3, user.id=7854aa333c774071aa916b8c294be9bf, scope.system=all) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:38:59.785860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3157c35-f22b-4f0c-a2ac-bc9a2fb53a2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:38:59.785860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3157c35-f22b-4f0c-a2ac-bc9a2fb53a2b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:38:59.802416 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 182/357] 10.222.0.60 () {56 vars in 1308 bytes} [Mon Sep 19 14:38:59 2022] GET /identity/v3/role_assignments?user.id=7854aa333c774071aa916b8c294be9bf&scope.system=all&role.id=2235d4074114441ea4661c3fb936dcc3 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:38:59.803847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3157c35-f22b-4f0c-a2ac-bc9a2fb53a2b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:59.830403 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf6607-6b8a-4225-b0db-440c616867e8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:59.879126 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d71c19-747f-43cb-9466-709613cabf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:59.881532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d71c19-747f-43cb-9466-709613cabf89 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:59.882066 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d71c19-747f-43cb-9466-709613cabf89 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:38:59.882552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d71c19-747f-43cb-9466-709613cabf89 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:38:59.896888 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d71c19-747f-43cb-9466-709613cabf89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-40d71c19-747f-43cb-9466-709613cabf89 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:38:59.902302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:38:59.903318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:38:59.904861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:38:59.904861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:38:59.904861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:38:59.904861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:38:59.904861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:38:59.904861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:38:59.904861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:38:59.904861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:38:59.904861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:38:59.904861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:38:59.904861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:38:59.904861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:38:59.904861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 176/358] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:38:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:38:59.904861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d71c19-747f-43cb-9466-709613cabf89 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:38:59.942826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3157c35-f22b-4f0c-a2ac-bc9a2fb53a2b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:38:59.982377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:38:59.999198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0664a2-d62c-430e-b6b7-1f006707a066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:38:59.999598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0664a2-d62c-430e-b6b7-1f006707a066 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:38:59.999909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0664a2-d62c-430e-b6b7-1f006707a066 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:00.000178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0664a2-d62c-430e-b6b7-1f006707a066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:00.000933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0664a2-d62c-430e-b6b7-1f006707a066 None admin] RBAC: Adding query filter params (group.id=2620063028584525a9ef3d5e557d96b6, role.id=0a6592b9a0b64df6938aba0a010c2256, scope.project.id=4681da4f1deb408293b2c948aec851ea) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:00.001241 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0664a2-d62c-430e-b6b7-1f006707a066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:00.002711 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0664a2-d62c-430e-b6b7-1f006707a066 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:00.009716 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 183/359] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Sep 19 14:38:59 2022] GET /identity/v3/role_assignments?group.id=2620063028584525a9ef3d5e557d96b6&scope.project.id=4681da4f1deb408293b2c948aec851ea&role.id=0a6592b9a0b64df6938aba0a010c2256 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:00.011191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2b909-1e4b-47a7-80ff-76bc7876a855 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:00.011606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2b909-1e4b-47a7-80ff-76bc7876a855 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:00.011987 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2b909-1e4b-47a7-80ff-76bc7876a855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:00.039491 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-31a2b909-1e4b-47a7-80ff-76bc7876a855 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:00.040775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-31a2b909-1e4b-47a7-80ff-76bc7876a855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:00.071368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d71c19-747f-43cb-9466-709613cabf89 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:00.095300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:00.126485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abadcab-7e4f-4508-be5d-7df10ae1cdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:00.127564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abadcab-7e4f-4508-be5d-7df10ae1cdfa None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:00.128128 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abadcab-7e4f-4508-be5d-7df10ae1cdfa None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:00.128625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abadcab-7e4f-4508-be5d-7df10ae1cdfa None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:00.130817 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abadcab-7e4f-4508-be5d-7df10ae1cdfa None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:00.132891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abadcab-7e4f-4508-be5d-7df10ae1cdfa None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:00.137132 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 177/360] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:38:59 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:00.138330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abadcab-7e4f-4508-be5d-7df10ae1cdfa None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:00.157262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 184/361] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:00.157262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2b909-1e4b-47a7-80ff-76bc7876a855 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:00.265086 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abadcab-7e4f-4508-be5d-7df10ae1cdfa None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:00.278714 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:00.301047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2b909-1e4b-47a7-80ff-76bc7876a855 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:00.316674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c1999-6151-4a48-9a07-30ee9c6a630a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:00.318142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c1999-6151-4a48-9a07-30ee9c6a630a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:00.318569 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c1999-6151-4a48-9a07-30ee9c6a630a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:00.318982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c1999-6151-4a48-9a07-30ee9c6a630a None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:00.319985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c1999-6151-4a48-9a07-30ee9c6a630a None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:00.320824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c1999-6151-4a48-9a07-30ee9c6a630a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:00.322811 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c1999-6151-4a48-9a07-30ee9c6a630a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:00.341335 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 178/362] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:39:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:00.342519 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c1999-6151-4a48-9a07-30ee9c6a630a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:00.344075 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dada26c-dfe0-4d3e-b125-f4bdf54cb761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:00.344820 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dada26c-dfe0-4d3e-b125-f4bdf54cb761 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:00.344982 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dada26c-dfe0-4d3e-b125-f4bdf54cb761 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:00.345252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dada26c-dfe0-4d3e-b125-f4bdf54cb761 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:00.363104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dada26c-dfe0-4d3e-b125-f4bdf54cb761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-5dada26c-dfe0-4d3e-b125-f4bdf54cb761 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:00.368125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:00.369295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:00.370384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:00.370384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:00.370384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:00.370384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:00.370384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:00.370384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:00.370384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:00.370384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:00.370384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:00.370384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:00.370384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:39:00.370384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:00.370384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 185/363] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:39:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:00.370384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dada26c-dfe0-4d3e-b125-f4bdf54cb761 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:00.416294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:00.479470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c1999-6151-4a48-9a07-30ee9c6a630a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:00.495303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dada26c-dfe0-4d3e-b125-f4bdf54cb761 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:00.525948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a87df-a6a1-4e5a-9f71-192a6ed49fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:00.526860 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a87df-a6a1-4e5a-9f71-192a6ed49fbc None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:00.527307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a87df-a6a1-4e5a-9f71-192a6ed49fbc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:00.527585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a87df-a6a1-4e5a-9f71-192a6ed49fbc None admin] PATH_INFO: `/v3/groups/2620063028584525a9ef3d5e557d96b6` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:00.530034 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a87df-a6a1-4e5a-9f71-192a6ed49fbc None admin] RBAC: Authorizing `identity:get_group(group_id=2620063028584525a9ef3d5e557d96b6)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:00.531474 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a87df-a6a1-4e5a-9f71-192a6ed49fbc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:00.533081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 179/364] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Sep 19 14:39:00 2022] GET /identity/v3/groups/2620063028584525a9ef3d5e557d96b6 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:00.535996 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a87df-a6a1-4e5a-9f71-192a6ed49fbc None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:00.540114 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b57331-ac7b-4114-8c2c-2ee1fa821f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:00.540727 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b57331-ac7b-4114-8c2c-2ee1fa821f41 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:00.541004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b57331-ac7b-4114-8c2c-2ee1fa821f41 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:00.541270 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b57331-ac7b-4114-8c2c-2ee1fa821f41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:00.583627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:00.631271 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b57331-ac7b-4114-8c2c-2ee1fa821f41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:00.632906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b57331-ac7b-4114-8c2c-2ee1fa821f41 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:00.662860 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a87df-a6a1-4e5a-9f71-192a6ed49fbc None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:00.674225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:00.714757 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630a258-eff5-4f72-ac33-8b1817129f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:00.715466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630a258-eff5-4f72-ac33-8b1817129f85 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:00.715811 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630a258-eff5-4f72-ac33-8b1817129f85 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:00.716629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630a258-eff5-4f72-ac33-8b1817129f85 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:00.717296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630a258-eff5-4f72-ac33-8b1817129f85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:00.717597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630a258-eff5-4f72-ac33-8b1817129f85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:00.718987 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630a258-eff5-4f72-ac33-8b1817129f85 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:00.728867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 180/365] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:00.730191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630a258-eff5-4f72-ac33-8b1817129f85 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:00.779995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 186/366] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:00.779995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b57331-ac7b-4114-8c2c-2ee1fa821f41 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:00.876745 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630a258-eff5-4f72-ac33-8b1817129f85 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:00.878805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:00.917479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b57331-ac7b-4114-8c2c-2ee1fa821f41 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:00.926678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0643ad9-d630-4c5b-bade-83ed9bb79c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:00.927355 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0643ad9-d630-4c5b-bade-83ed9bb79c61 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:00.927596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0643ad9-d630-4c5b-bade-83ed9bb79c61 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:00.927958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0643ad9-d630-4c5b-bade-83ed9bb79c61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:00.940736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0643ad9-d630-4c5b-bade-83ed9bb79c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-f0643ad9-d630-4c5b-bade-83ed9bb79c61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:00.946145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:00.947076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:00.947967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:00.947967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:00.947967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:00.947967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:00.947967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:00.947967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:00.947967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:00.947967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:00.947967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:00.947967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:00.947967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:00.947967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:00.947967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 181/367] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:39:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:00.948653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0643ad9-d630-4c5b-bade-83ed9bb79c61 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:00.954436 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:00.970918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176516c3-3f87-4f35-bfd0-06802a2dd9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:00.971540 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176516c3-3f87-4f35-bfd0-06802a2dd9a2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:00.971850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176516c3-3f87-4f35-bfd0-06802a2dd9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:00.972119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176516c3-3f87-4f35-bfd0-06802a2dd9a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:00.972852 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176516c3-3f87-4f35-bfd0-06802a2dd9a2 None admin] RBAC: Adding query filter params (group.id=2620063028584525a9ef3d5e557d96b6, role.id=fe3bbad209634f16b1fcb66d6d49f665, scope.project.id=4681da4f1deb408293b2c948aec851ea) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:00.973159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176516c3-3f87-4f35-bfd0-06802a2dd9a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:00.976769 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176516c3-3f87-4f35-bfd0-06802a2dd9a2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:00.985485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 187/368] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Sep 19 14:39:00 2022] GET /identity/v3/role_assignments?group.id=2620063028584525a9ef3d5e557d96b6&scope.project.id=4681da4f1deb408293b2c948aec851ea&role.id=fe3bbad209634f16b1fcb66d6d49f665 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:00.986324 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176516c3-3f87-4f35-bfd0-06802a2dd9a2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:01.121720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176516c3-3f87-4f35-bfd0-06802a2dd9a2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:01.124622 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0643ad9-d630-4c5b-bade-83ed9bb79c61 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:01.173054 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4e881-7ec2-4ef3-a3e8-b90437780090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:01.173054 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4e881-7ec2-4ef3-a3e8-b90437780090 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:01.173054 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4e881-7ec2-4ef3-a3e8-b90437780090 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:01.173054 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4e881-7ec2-4ef3-a3e8-b90437780090 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:01.175051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4e881-7ec2-4ef3-a3e8-b90437780090 None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:01.176562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4e881-7ec2-4ef3-a3e8-b90437780090 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:01.178914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 188/369] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:39:00 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:01.179727 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4e881-7ec2-4ef3-a3e8-b90437780090 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:01.185494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff8b1e-a517-46d4-b661-9807eddbb81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:01.186415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff8b1e-a517-46d4-b661-9807eddbb81e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:01.186694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff8b1e-a517-46d4-b661-9807eddbb81e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:01.187416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cff8b1e-a517-46d4-b661-9807eddbb81e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:01.229136 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:01.274230 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:01.285921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff8b1e-a517-46d4-b661-9807eddbb81e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:01.287408 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cff8b1e-a517-46d4-b661-9807eddbb81e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:01.333060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4e881-7ec2-4ef3-a3e8-b90437780090 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:01.379980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bd73b-55b0-4962-a48c-b836deac5fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:01.380577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bd73b-55b0-4962-a48c-b836deac5fa6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:01.380800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bd73b-55b0-4962-a48c-b836deac5fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:01.381058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bd73b-55b0-4962-a48c-b836deac5fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:01.382349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bd73b-55b0-4962-a48c-b836deac5fa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:01.382654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bd73b-55b0-4962-a48c-b836deac5fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:01.384521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bd73b-55b0-4962-a48c-b836deac5fa6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:01.400802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 189/370] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:01.401520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bd73b-55b0-4962-a48c-b836deac5fa6 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:01.415981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 182/371] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:01.418375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff8b1e-a517-46d4-b661-9807eddbb81e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:01.488660 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:01.523076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bd73b-55b0-4962-a48c-b836deac5fa6 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:01.534963 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cff8b1e-a517-46d4-b661-9807eddbb81e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:01.558503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e98b9d-399d-46a4-bbfb-743757fe60f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:01.559141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e98b9d-399d-46a4-bbfb-743757fe60f5 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:01.559450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e98b9d-399d-46a4-bbfb-743757fe60f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:01.559734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e98b9d-399d-46a4-bbfb-743757fe60f5 None admin] PATH_INFO: `/v3/groups/2620063028584525a9ef3d5e557d96b6` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:01.561909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e98b9d-399d-46a4-bbfb-743757fe60f5 None admin] RBAC: Authorizing `identity:get_group(group_id=2620063028584525a9ef3d5e557d96b6)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:01.563373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e98b9d-399d-46a4-bbfb-743757fe60f5 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:01.565219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 190/372] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Sep 19 14:39:01 2022] GET /identity/v3/groups/2620063028584525a9ef3d5e557d96b6 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:01.565820 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e98b9d-399d-46a4-bbfb-743757fe60f5 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:01.566490 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:01.572740 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094eb92-be4b-47eb-9f7b-6fd2dd1c98c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:01.573386 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094eb92-be4b-47eb-9f7b-6fd2dd1c98c7 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:01.573712 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094eb92-be4b-47eb-9f7b-6fd2dd1c98c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:01.573921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094eb92-be4b-47eb-9f7b-6fd2dd1c98c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:01.652973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094eb92-be4b-47eb-9f7b-6fd2dd1c98c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:01.655191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094eb92-be4b-47eb-9f7b-6fd2dd1c98c7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:01.701520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e98b9d-399d-46a4-bbfb-743757fe60f5 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:01.725101 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:01.749282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec1131-eab6-4498-b3ec-381033e3b350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:01.749934 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec1131-eab6-4498-b3ec-381033e3b350 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:01.750214 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec1131-eab6-4498-b3ec-381033e3b350 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:01.750763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec1131-eab6-4498-b3ec-381033e3b350 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:01.752050 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec1131-eab6-4498-b3ec-381033e3b350 None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:01.753642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec1131-eab6-4498-b3ec-381033e3b350 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:01.756565 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 191/373] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:39:01 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:01.757418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec1131-eab6-4498-b3ec-381033e3b350 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:01.770314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 183/374] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:01.771625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094eb92-be4b-47eb-9f7b-6fd2dd1c98c7 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:01.828140 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:01.886270 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec1131-eab6-4498-b3ec-381033e3b350 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:01.894031 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094eb92-be4b-47eb-9f7b-6fd2dd1c98c7 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:01.933321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f7779-edc7-47ba-a587-5e1327c7a97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:01.933856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f7779-edc7-47ba-a587-5e1327c7a97e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:01.934145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f7779-edc7-47ba-a587-5e1327c7a97e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:01.934442 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f7779-edc7-47ba-a587-5e1327c7a97e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:01.935646 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd52f54-3558-42c2-9914-90d07024bcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:01.935765 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f7779-edc7-47ba-a587-5e1327c7a97e None admin] RBAC: Adding query filter params (group.id=2620063028584525a9ef3d5e557d96b6, role.id=0a6592b9a0b64df6938aba0a010c2256, scope.project.id=17924018ca584f3696136b838ab2fdc8) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:01.935990 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f7779-edc7-47ba-a587-5e1327c7a97e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:01.936825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd52f54-3558-42c2-9914-90d07024bcde None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:01.937307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd52f54-3558-42c2-9914-90d07024bcde None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:01.937409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f7779-edc7-47ba-a587-5e1327c7a97e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:01.937873 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd52f54-3558-42c2-9914-90d07024bcde None admin] PATH_INFO: `/v3/groups/2620063028584525a9ef3d5e557d96b6` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:01.939295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd52f54-3558-42c2-9914-90d07024bcde None admin] RBAC: Authorizing `identity:get_group(group_id=2620063028584525a9ef3d5e557d96b6)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:01.941051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd52f54-3558-42c2-9914-90d07024bcde None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:01.942891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 184/375] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Sep 19 14:39:01 2022] GET /identity/v3/groups/2620063028584525a9ef3d5e557d96b6 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:01.944068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd52f54-3558-42c2-9914-90d07024bcde None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:01.944339 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 192/376] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Sep 19 14:39:01 2022] GET /identity/v3/role_assignments?group.id=2620063028584525a9ef3d5e557d96b6&scope.project.id=17924018ca584f3696136b838ab2fdc8&role.id=0a6592b9a0b64df6938aba0a010c2256 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:01.951672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f7779-edc7-47ba-a587-5e1327c7a97e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:01.999898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:02.131923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd52f54-3558-42c2-9914-90d07024bcde None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:02.140827 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f7779-edc7-47ba-a587-5e1327c7a97e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:02.170300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:02.182736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbabbe2-7ee5-44cd-b467-b89a1f6333f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:02.183718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbabbe2-7ee5-44cd-b467-b89a1f6333f5 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:02.184185 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbabbe2-7ee5-44cd-b467-b89a1f6333f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:02.185228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbabbe2-7ee5-44cd-b467-b89a1f6333f5 None admin] PATH_INFO: `/v3/projects/b206fdda52da4f339c6c008d4c7973af` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:02.186474 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbabbe2-7ee5-44cd-b467-b89a1f6333f5 None admin] RBAC: Authorizing `identity:get_project(project_id=b206fdda52da4f339c6c008d4c7973af)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:02.188014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbabbe2-7ee5-44cd-b467-b89a1f6333f5 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:02.189719 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177c181-1020-4e78-95c3-71cd1e7acce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:02.190344 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177c181-1020-4e78-95c3-71cd1e7acce3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:02.190511 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 185/377] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:39:01 2022] GET /identity/v3/projects/b206fdda52da4f339c6c008d4c7973af => generated 344 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:02.190791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177c181-1020-4e78-95c3-71cd1e7acce3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:02.191093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177c181-1020-4e78-95c3-71cd1e7acce3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:02.191813 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177c181-1020-4e78-95c3-71cd1e7acce3 None admin] RBAC: Adding query filter params (group.id=2620063028584525a9ef3d5e557d96b6, role.id=fe3bbad209634f16b1fcb66d6d49f665, scope.project.id=17924018ca584f3696136b838ab2fdc8) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:02.195174 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177c181-1020-4e78-95c3-71cd1e7acce3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:02.195174 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177c181-1020-4e78-95c3-71cd1e7acce3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:02.195892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbabbe2-7ee5-44cd-b467-b89a1f6333f5 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:02.210155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 193/378] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Sep 19 14:39:01 2022] GET /identity/v3/role_assignments?group.id=2620063028584525a9ef3d5e557d96b6&scope.project.id=17924018ca584f3696136b838ab2fdc8&role.id=fe3bbad209634f16b1fcb66d6d49f665 => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:02.319125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:02.323544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbabbe2-7ee5-44cd-b467-b89a1f6333f5 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:02.375021 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda3d75-ac87-44c2-9b9a-05b55ef886b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:02.375686 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda3d75-ac87-44c2-9b9a-05b55ef886b4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:02.376002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda3d75-ac87-44c2-9b9a-05b55ef886b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:02.376283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda3d75-ac87-44c2-9b9a-05b55ef886b4 None admin] PATH_INFO: `/v3/groups/a160a734043a4483988bd33da3922f3e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:02.391544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda3d75-ac87-44c2-9b9a-05b55ef886b4 None admin] RBAC: Authorizing `identity:get_group(group_id=a160a734043a4483988bd33da3922f3e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:02.393113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda3d75-ac87-44c2-9b9a-05b55ef886b4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:02.394849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 186/379] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Sep 19 14:39:02 2022] GET /identity/v3/groups/a160a734043a4483988bd33da3922f3e => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:02.401156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177c181-1020-4e78-95c3-71cd1e7acce3 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:02.521783 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177c181-1020-4e78-95c3-71cd1e7acce3 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:02.565048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27fe91-0cd8-4109-9853-a25d7dfbbf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:02.566067 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27fe91-0cd8-4109-9853-a25d7dfbbf30 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:02.566551 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27fe91-0cd8-4109-9853-a25d7dfbbf30 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:02.567043 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27fe91-0cd8-4109-9853-a25d7dfbbf30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:02.568038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27fe91-0cd8-4109-9853-a25d7dfbbf30 None admin] RBAC: Adding query filter params (group.id=a160a734043a4483988bd33da3922f3e, role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=b206fdda52da4f339c6c008d4c7973af) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:02.568572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27fe91-0cd8-4109-9853-a25d7dfbbf30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:02.570462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27fe91-0cd8-4109-9853-a25d7dfbbf30 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:02.577878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 194/380] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Sep 19 14:39:02 2022] GET /identity/v3/role_assignments?group.id=a160a734043a4483988bd33da3922f3e&scope.project.id=b206fdda52da4f339c6c008d4c7973af&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:02.702580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a0d66-1520-452f-bb3d-7b95aadc4c51 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:02.703098 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a0d66-1520-452f-bb3d-7b95aadc4c51 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:02.703174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a0d66-1520-452f-bb3d-7b95aadc4c51 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:02.704592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 187/381] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:02.710341 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d77e19-ffa2-4d55-9281-dde512338683 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:02.710642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d77e19-ffa2-4d55-9281-dde512338683 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:02.710904 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d77e19-ffa2-4d55-9281-dde512338683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:02.734282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c3d77e19-ffa2-4d55-9281-dde512338683 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:02.735502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c3d77e19-ffa2-4d55-9281-dde512338683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:02.813309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:02.842010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 195/382] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:03.162762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a0d66-1520-452f-bb3d-7b95aadc4c51 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:03.297743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a0d66-1520-452f-bb3d-7b95aadc4c51 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:03.340534 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb2feb-e2e9-4435-9d5c-a3d5fe30f555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:03.342028 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb2feb-e2e9-4435-9d5c-a3d5fe30f555 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:03.342523 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb2feb-e2e9-4435-9d5c-a3d5fe30f555 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:03.342993 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb2feb-e2e9-4435-9d5c-a3d5fe30f555 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:03.361928 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb2feb-e2e9-4435-9d5c-a3d5fe30f555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-bebb2feb-e2e9-4435-9d5c-a3d5fe30f555 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:03.367068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:03.367986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:03.368840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:03.368840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:03.368840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:03.368840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:03.368840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:03.368840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:03.368840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:03.368840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:03.368840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:03.368840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:03.368840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:39:03.368840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:03.372765 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 188/383] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:03.377552 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d77e19-ffa2-4d55-9281-dde512338683 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:03.525164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d77e19-ffa2-4d55-9281-dde512338683 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:03.562649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdad389-cd52-441f-9f30-63f55a8ca729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:03.563612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdad389-cd52-441f-9f30-63f55a8ca729 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:03.564120 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdad389-cd52-441f-9f30-63f55a8ca729 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:03.564614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdad389-cd52-441f-9f30-63f55a8ca729 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:03.565485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdad389-cd52-441f-9f30-63f55a8ca729 None admin] RBAC: Adding query filter params (name=member) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:03.565994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdad389-cd52-441f-9f30-63f55a8ca729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:03.567536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdad389-cd52-441f-9f30-63f55a8ca729 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:03.579716 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 196/384] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:39:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:03.587362 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb2feb-e2e9-4435-9d5c-a3d5fe30f555 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:03.613304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:03.711302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb2feb-e2e9-4435-9d5c-a3d5fe30f555 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:03.753805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145286b-7fa7-4c19-bda0-b603c262f72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:03.754851 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145286b-7fa7-4c19-bda0-b603c262f72b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:03.755384 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145286b-7fa7-4c19-bda0-b603c262f72b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:03.755903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145286b-7fa7-4c19-bda0-b603c262f72b None admin] PATH_INFO: `/v3/groups/2620063028584525a9ef3d5e557d96b6` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:03.760728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145286b-7fa7-4c19-bda0-b603c262f72b None admin] RBAC: Authorizing `identity:get_group(group_id=2620063028584525a9ef3d5e557d96b6)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:03.763185 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145286b-7fa7-4c19-bda0-b603c262f72b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:03.765124 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 189/385] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Sep 19 14:39:03 2022] GET /identity/v3/groups/2620063028584525a9ef3d5e557d96b6 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:03.771220 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdad389-cd52-441f-9f30-63f55a8ca729 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:03.822969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:03.891944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdad389-cd52-441f-9f30-63f55a8ca729 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:03.928947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c292cbf-e60e-45a3-b7ed-7c618b2a4a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:03.929937 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c292cbf-e60e-45a3-b7ed-7c618b2a4a18 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:03.930417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c292cbf-e60e-45a3-b7ed-7c618b2a4a18 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:03.930917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c292cbf-e60e-45a3-b7ed-7c618b2a4a18 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:03.932275 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c292cbf-e60e-45a3-b7ed-7c618b2a4a18 None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:03.933932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c292cbf-e60e-45a3-b7ed-7c618b2a4a18 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:03.939922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 197/386] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:39:03 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:03.942606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145286b-7fa7-4c19-bda0-b603c262f72b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:03.992997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3e85f-d483-4470-b01f-660a8da0c16f None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:03.993646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3e85f-d483-4470-b01f-660a8da0c16f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:03.994122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3e85f-d483-4470-b01f-660a8da0c16f None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:03.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 198/387] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:39:04.001137 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29171d5-cf10-4732-98d9-775ae79a81be None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:04.001636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29171d5-cf10-4732-98d9-775ae79a81be None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:04.002157 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29171d5-cf10-4732-98d9-775ae79a81be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:04.029646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c29171d5-cf10-4732-98d9-775ae79a81be None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:04.032356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c29171d5-cf10-4732-98d9-775ae79a81be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:04.124211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:04.136999 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145286b-7fa7-4c19-bda0-b603c262f72b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:04.186339 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 199/388] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:39:04.193136 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2198227-ac20-43f6-a603-bc1552309357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:04.194047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2198227-ac20-43f6-a603-bc1552309357 None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:04.194542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2198227-ac20-43f6-a603-bc1552309357 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:04.195048 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2198227-ac20-43f6-a603-bc1552309357 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea/groups/2620063028584525a9ef3d5e557d96b6/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:04.203845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2198227-ac20-43f6-a603-bc1552309357 None admin] RBAC: Authorizing `identity:create_grant(project_id=4681da4f1deb408293b2c948aec851ea, group_id=2620063028584525a9ef3d5e557d96b6, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:04.205639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2198227-ac20-43f6-a603-bc1552309357 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:04.224360 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 190/389] 10.222.0.60 () {58 vars in 1442 bytes} [Mon Sep 19 14:39:03 2022] PUT /identity/v3/projects/4681da4f1deb408293b2c948aec851ea/groups/2620063028584525a9ef3d5e557d96b6/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:39:04.555495 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29171d5-cf10-4732-98d9-775ae79a81be None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:04.607756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:04.675733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29171d5-cf10-4732-98d9-775ae79a81be None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:04.713298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b861d3-da8c-44d1-86f8-0e53bf2abbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:04.714302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b861d3-da8c-44d1-86f8-0e53bf2abbd3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:04.714784 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b861d3-da8c-44d1-86f8-0e53bf2abbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:04.715279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b861d3-da8c-44d1-86f8-0e53bf2abbd3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:04.727874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b861d3-da8c-44d1-86f8-0e53bf2abbd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-f6b861d3-da8c-44d1-86f8-0e53bf2abbd3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:04.733708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:04.734598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:04.735417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:04.735417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:04.735417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:04.735417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:04.735417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:04.735417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:04.735417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:04.735417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:04.735417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:04.735417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:04.735417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:39:04.735417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:04.742326 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 200/390] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:39:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:04.745117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2198227-ac20-43f6-a603-bc1552309357 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:04.756225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059594bd-fbcd-47b3-ab6c-87c341d5e7e4 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:04.756886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059594bd-fbcd-47b3-ab6c-87c341d5e7e4 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:04.757951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059594bd-fbcd-47b3-ab6c-87c341d5e7e4 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:04.759766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 201/391] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:39:04.769672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac7842-6886-44f4-be8d-fc19da4cbd55 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:04.770554 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac7842-6886-44f4-be8d-fc19da4cbd55 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:04.771038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac7842-6886-44f4-be8d-fc19da4cbd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:04.791816 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9bac7842-6886-44f4-be8d-fc19da4cbd55 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:04.793441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9bac7842-6886-44f4-be8d-fc19da4cbd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:04.877911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 202/392] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:39:04.879592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3462f-2514-4ab0-a505-ebc2afc40a08 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:04.880111 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3462f-2514-4ab0-a505-ebc2afc40a08 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:04.880667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3462f-2514-4ab0-a505-ebc2afc40a08 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:04.882439 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 203/393] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:04.889146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d1860-6869-4b13-b7ea-4db56be03df2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:04.889629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d1860-6869-4b13-b7ea-4db56be03df2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:04.890131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d1860-6869-4b13-b7ea-4db56be03df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:04.928061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-584d1860-6869-4b13-b7ea-4db56be03df2 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:04.932694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-584d1860-6869-4b13-b7ea-4db56be03df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:04.974107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2198227-ac20-43f6-a603-bc1552309357 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:04.985570 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:05.042705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d008a30-0900-47fb-9252-11ead103911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:05.043653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d008a30-0900-47fb-9252-11ead103911d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:05.044164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d008a30-0900-47fb-9252-11ead103911d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:05.044618 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d008a30-0900-47fb-9252-11ead103911d None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:05.045480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d008a30-0900-47fb-9252-11ead103911d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:05.046004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d008a30-0900-47fb-9252-11ead103911d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:05.048633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d008a30-0900-47fb-9252-11ead103911d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:05.063718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 191/394] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:39:05.075929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d008a30-0900-47fb-9252-11ead103911d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:05.080940 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 204/395] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:05.104068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:05.221759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d008a30-0900-47fb-9252-11ead103911d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:05.261780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7deca-febd-424d-87d4-3756c611340b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:05.263324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7deca-febd-424d-87d4-3756c611340b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:05.263854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7deca-febd-424d-87d4-3756c611340b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:05.264329 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7deca-febd-424d-87d4-3756c611340b None admin] PATH_INFO: `/v3/groups/2620063028584525a9ef3d5e557d96b6` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:05.265631 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7deca-febd-424d-87d4-3756c611340b None admin] RBAC: Authorizing `identity:get_group(group_id=2620063028584525a9ef3d5e557d96b6)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:05.267418 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7deca-febd-424d-87d4-3756c611340b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:05.270561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 192/396] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Sep 19 14:39:05 2022] GET /identity/v3/groups/2620063028584525a9ef3d5e557d96b6 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:05.276225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d1860-6869-4b13-b7ea-4db56be03df2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:05.361228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7deca-febd-424d-87d4-3756c611340b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:05.460201 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d1860-6869-4b13-b7ea-4db56be03df2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:05.502950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fc5f8-a20a-44be-862b-82f501d84d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:05.503510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fc5f8-a20a-44be-862b-82f501d84d58 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:05.503793 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fc5f8-a20a-44be-862b-82f501d84d58 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:05.504105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067fc5f8-a20a-44be-862b-82f501d84d58 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:05.505260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067fc5f8-a20a-44be-862b-82f501d84d58 None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:05.506769 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067fc5f8-a20a-44be-862b-82f501d84d58 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:05.509317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 205/397] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:39:05 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:05.510100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fc5f8-a20a-44be-862b-82f501d84d58 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:05.521541 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:05.547695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7deca-febd-424d-87d4-3756c611340b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:05.559374 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:05.597109 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235bbae-28d1-43db-b892-c98e1864e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:05.598012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235bbae-28d1-43db-b892-c98e1864e39b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:05.598299 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235bbae-28d1-43db-b892-c98e1864e39b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:05.598574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3235bbae-28d1-43db-b892-c98e1864e39b None admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:05.612307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3235bbae-28d1-43db-b892-c98e1864e39b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-3235bbae-28d1-43db-b892-c98e1864e39b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:05.620473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:05.621479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:05.622819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:05.622819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:05.622819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:05.622819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:05.622819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:05.622819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:05.622819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:05.622819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:05.622819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:05.622819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:05.622819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:39:05.622819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:05.623703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 193/398] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:05.628283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235bbae-28d1-43db-b892-c98e1864e39b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:05.693522 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067fc5f8-a20a-44be-862b-82f501d84d58 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:05.736133 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90f66a-8254-4224-8b9e-81e70ab861cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:05.736785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90f66a-8254-4224-8b9e-81e70ab861cf None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:05.737060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90f66a-8254-4224-8b9e-81e70ab861cf None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:05.737333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90f66a-8254-4224-8b9e-81e70ab861cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:05.750153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90f66a-8254-4224-8b9e-81e70ab861cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-da90f66a-8254-4224-8b9e-81e70ab861cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:05.755229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:05.756988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:05.758634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:05.758634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:05.758634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:05.758634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:05.758634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:05.758634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:05.758634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:05.758634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:05.758634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:05.758634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:05.758634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:39:05.758634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:05.764504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 206/399] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:39:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:05.764504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90f66a-8254-4224-8b9e-81e70ab861cf None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:05.793224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3235bbae-28d1-43db-b892-c98e1864e39b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:05.860231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9309d-8314-4f1d-b009-61e674e502a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:05.861277 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:05.861471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9309d-8314-4f1d-b009-61e674e502a0 None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:05.861824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9309d-8314-4f1d-b009-61e674e502a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:05.862320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9309d-8314-4f1d-b009-61e674e502a0 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea/groups/2620063028584525a9ef3d5e557d96b6/roles/fe3bbad209634f16b1fcb66d6d49f665` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:05.865818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9309d-8314-4f1d-b009-61e674e502a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4681da4f1deb408293b2c948aec851ea, group_id=2620063028584525a9ef3d5e557d96b6, role_id=fe3bbad209634f16b1fcb66d6d49f665)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:05.868702 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c9309d-8314-4f1d-b009-61e674e502a0 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:05.880260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90f66a-8254-4224-8b9e-81e70ab861cf None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:05.886815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:05.896065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 194/400] 10.222.0.60 () {58 vars in 1442 bytes} [Mon Sep 19 14:39:05 2022] PUT /identity/v3/projects/4681da4f1deb408293b2c948aec851ea/groups/2620063028584525a9ef3d5e557d96b6/roles/fe3bbad209634f16b1fcb66d6d49f665 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:39:05.896258 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9309d-8314-4f1d-b009-61e674e502a0 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:05.933590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73674315-cbde-4b87-a07e-65bd3e5e8e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:05.934540 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73674315-cbde-4b87-a07e-65bd3e5e8e0c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:05.934957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73674315-cbde-4b87-a07e-65bd3e5e8e0c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:05.935474 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73674315-cbde-4b87-a07e-65bd3e5e8e0c None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:05.936318 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73674315-cbde-4b87-a07e-65bd3e5e8e0c None admin] RBAC: Adding query filter params (name=member) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:05.936818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73674315-cbde-4b87-a07e-65bd3e5e8e0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:05.938430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73674315-cbde-4b87-a07e-65bd3e5e8e0c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:05.948683 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 207/401] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:39:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:05.950922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15455073-3fb1-41c0-b488-df73ef9eea59 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:05.951356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15455073-3fb1-41c0-b488-df73ef9eea59 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:05.951764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15455073-3fb1-41c0-b488-df73ef9eea59 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:05.953252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 208/402] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:05.963791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4140d0b-6cdd-48ec-8a59-fd5eef0d3ae9 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:05.963791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4140d0b-6cdd-48ec-8a59-fd5eef0d3ae9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:05.963791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4140d0b-6cdd-48ec-8a59-fd5eef0d3ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:05.998161 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e4140d0b-6cdd-48ec-8a59-fd5eef0d3ae9 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:06.000350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e4140d0b-6cdd-48ec-8a59-fd5eef0d3ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:06.057877 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9309d-8314-4f1d-b009-61e674e502a0 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:06.114441 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119911a-b0ee-4689-85ff-a5f3211edd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:06.115051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119911a-b0ee-4689-85ff-a5f3211edd65 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:06.115324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119911a-b0ee-4689-85ff-a5f3211edd65 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:06.115594 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d119911a-b0ee-4689-85ff-a5f3211edd65 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:06.116389 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d119911a-b0ee-4689-85ff-a5f3211edd65 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:06.117081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d119911a-b0ee-4689-85ff-a5f3211edd65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:06.119417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d119911a-b0ee-4689-85ff-a5f3211edd65 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:06.119912 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 209/403] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:39:06.121100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4140d0b-6cdd-48ec-8a59-fd5eef0d3ae9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:06.129645 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 195/404] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:06.135075 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119911a-b0ee-4689-85ff-a5f3211edd65 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:06.212981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:06.243847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:06.259375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d119911a-b0ee-4689-85ff-a5f3211edd65 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:06.305313 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4140d0b-6cdd-48ec-8a59-fd5eef0d3ae9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:06.310493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12b197-4b69-4066-ba2b-3aaeef8f678b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:06.311390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12b197-4b69-4066-ba2b-3aaeef8f678b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:06.311872 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12b197-4b69-4066-ba2b-3aaeef8f678b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:06.312348 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12b197-4b69-4066-ba2b-3aaeef8f678b None admin] PATH_INFO: `/v3/groups/2620063028584525a9ef3d5e557d96b6` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:06.320637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12b197-4b69-4066-ba2b-3aaeef8f678b None admin] RBAC: Authorizing `identity:get_group(group_id=2620063028584525a9ef3d5e557d96b6)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:06.322192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12b197-4b69-4066-ba2b-3aaeef8f678b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:06.324125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 196/405] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Sep 19 14:39:06 2022] GET /identity/v3/groups/2620063028584525a9ef3d5e557d96b6 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:06.329762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12b197-4b69-4066-ba2b-3aaeef8f678b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:06.378874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d78925-447c-4953-a504-04f3093669d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:06.381067 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d78925-447c-4953-a504-04f3093669d3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:06.381653 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d78925-447c-4953-a504-04f3093669d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:06.385664 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d78925-447c-4953-a504-04f3093669d3 None admin] PATH_INFO: `/v3/groups/2620063028584525a9ef3d5e557d96b6` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:06.387300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d78925-447c-4953-a504-04f3093669d3 None admin] RBAC: Authorizing `identity:get_group(group_id=2620063028584525a9ef3d5e557d96b6)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:06.389248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d78925-447c-4953-a504-04f3093669d3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:06.397488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 210/406] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Sep 19 14:39:06 2022] GET /identity/v3/groups/2620063028584525a9ef3d5e557d96b6 => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:06.401675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d78925-447c-4953-a504-04f3093669d3 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:06.492133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12b197-4b69-4066-ba2b-3aaeef8f678b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:06.522871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:06.529847 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86272cbc-0429-4037-96f6-844875333266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:06.530768 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86272cbc-0429-4037-96f6-844875333266 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:06.531263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86272cbc-0429-4037-96f6-844875333266 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:06.531720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86272cbc-0429-4037-96f6-844875333266 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:06.533070 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86272cbc-0429-4037-96f6-844875333266 None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:06.534696 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86272cbc-0429-4037-96f6-844875333266 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:06.536549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 197/407] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:39:06 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:06.537572 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86272cbc-0429-4037-96f6-844875333266 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:06.544619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d78925-447c-4953-a504-04f3093669d3 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:06.587732 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd84c20-eb4c-48a8-b37e-a8d6dc9c8c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:06.589187 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd84c20-eb4c-48a8-b37e-a8d6dc9c8c08 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:06.589646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd84c20-eb4c-48a8-b37e-a8d6dc9c8c08 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:06.590118 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd84c20-eb4c-48a8-b37e-a8d6dc9c8c08 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:06.592033 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd84c20-eb4c-48a8-b37e-a8d6dc9c8c08 None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:06.593656 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd84c20-eb4c-48a8-b37e-a8d6dc9c8c08 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:06.600410 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 211/408] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:39:06 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:06.601450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd84c20-eb4c-48a8-b37e-a8d6dc9c8c08 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:06.628218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:06.662969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86272cbc-0429-4037-96f6-844875333266 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:06.709559 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca39794-14ec-4c21-bc12-28c1579f4646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:06.710322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca39794-14ec-4c21-bc12-28c1579f4646 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:06.710603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca39794-14ec-4c21-bc12-28c1579f4646 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:06.710878 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca39794-14ec-4c21-bc12-28c1579f4646 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:06.719930 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd84c20-eb4c-48a8-b37e-a8d6dc9c8c08 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:06.728741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca39794-14ec-4c21-bc12-28c1579f4646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-4ca39794-14ec-4c21-bc12-28c1579f4646 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:06.735904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:06.736868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:06.737688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:06.737688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:06.737688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:06.737688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:06.737688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:06.737688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:06.737688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:06.737688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:06.737688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:06.737688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:06.737688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:06.737688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:06.737688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 198/409] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:39:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:06.738366 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca39794-14ec-4c21-bc12-28c1579f4646 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:06.765665 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24ec8d-cb26-4ded-be4f-112242935ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:06.766716 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24ec8d-cb26-4ded-be4f-112242935ac3 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:06.767307 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24ec8d-cb26-4ded-be4f-112242935ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:06.767846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24ec8d-cb26-4ded-be4f-112242935ac3 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8/groups/2620063028584525a9ef3d5e557d96b6/roles/fe3bbad209634f16b1fcb66d6d49f665` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:06.771374 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24ec8d-cb26-4ded-be4f-112242935ac3 None admin] RBAC: Authorizing `identity:create_grant(project_id=17924018ca584f3696136b838ab2fdc8, group_id=2620063028584525a9ef3d5e557d96b6, role_id=fe3bbad209634f16b1fcb66d6d49f665)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:06.773716 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24ec8d-cb26-4ded-be4f-112242935ac3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:06.795655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 212/410] 10.222.0.60 () {58 vars in 1442 bytes} [Mon Sep 19 14:39:06 2022] PUT /identity/v3/projects/17924018ca584f3696136b838ab2fdc8/groups/2620063028584525a9ef3d5e557d96b6/roles/fe3bbad209634f16b1fcb66d6d49f665 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:39:06.798224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24ec8d-cb26-4ded-be4f-112242935ac3 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:06.812242 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:06.874364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca39794-14ec-4c21-bc12-28c1579f4646 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:06.919690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e9dd0-6b41-4402-a04e-5510c4115aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:06.920782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e9dd0-6b41-4402-a04e-5510c4115aa3 None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:06.921207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e9dd0-6b41-4402-a04e-5510c4115aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:06.921903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e9dd0-6b41-4402-a04e-5510c4115aa3 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8/groups/2620063028584525a9ef3d5e557d96b6/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:06.924665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e9dd0-6b41-4402-a04e-5510c4115aa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=17924018ca584f3696136b838ab2fdc8, group_id=2620063028584525a9ef3d5e557d96b6, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:06.927004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e9dd0-6b41-4402-a04e-5510c4115aa3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:06.953519 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 199/411] 10.222.0.60 () {58 vars in 1442 bytes} [Mon Sep 19 14:39:06 2022] PUT /identity/v3/projects/17924018ca584f3696136b838ab2fdc8/groups/2620063028584525a9ef3d5e557d96b6/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:39:06.956185 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:06.991668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24ec8d-cb26-4ded-be4f-112242935ac3 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:07.054007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c930972-9375-45a9-bc2b-fe3a9f6a4ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:07.055003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c930972-9375-45a9-bc2b-fe3a9f6a4ad6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:07.055430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c930972-9375-45a9-bc2b-fe3a9f6a4ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:07.055863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c930972-9375-45a9-bc2b-fe3a9f6a4ad6 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:07.056705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c930972-9375-45a9-bc2b-fe3a9f6a4ad6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:07.057148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c930972-9375-45a9-bc2b-fe3a9f6a4ad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:07.058679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c930972-9375-45a9-bc2b-fe3a9f6a4ad6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:07.069646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 213/412] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:07.077014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e9dd0-6b41-4402-a04e-5510c4115aa3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:07.221699 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e9dd0-6b41-4402-a04e-5510c4115aa3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:07.245339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:07.265016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbd985-1693-4727-9de8-0748339f7c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:07.265650 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbd985-1693-4727-9de8-0748339f7c63 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:07.265929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbd985-1693-4727-9de8-0748339f7c63 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:07.266201 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbd985-1693-4727-9de8-0748339f7c63 None admin] PATH_INFO: `/v3/groups/a160a734043a4483988bd33da3922f3e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:07.267552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbd985-1693-4727-9de8-0748339f7c63 None admin] RBAC: Authorizing `identity:get_group(group_id=a160a734043a4483988bd33da3922f3e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:07.269078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bbd985-1693-4727-9de8-0748339f7c63 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:07.270957 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 200/413] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Sep 19 14:39:07 2022] GET /identity/v3/groups/a160a734043a4483988bd33da3922f3e => generated 229 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:07.276622 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c930972-9375-45a9-bc2b-fe3a9f6a4ad6 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:07.305533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754fa0a-b135-478a-b600-3b2f4006779c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:07.305835 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754fa0a-b135-478a-b600-3b2f4006779c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:07.306077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754fa0a-b135-478a-b600-3b2f4006779c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:07.307656 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 201/414] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:39:07.313200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7704f0a-341c-49b3-a82a-07ab30e5790e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:07.313603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7704f0a-341c-49b3-a82a-07ab30e5790e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:07.314028 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7704f0a-341c-49b3-a82a-07ab30e5790e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:07.336393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d7704f0a-341c-49b3-a82a-07ab30e5790e None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:07.338340 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d7704f0a-341c-49b3-a82a-07ab30e5790e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:07.418711 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c930972-9375-45a9-bc2b-fe3a9f6a4ad6 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:07.461063 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6dfee-4ff6-4578-a161-3f6bb96d2246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:07.462100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6dfee-4ff6-4578-a161-3f6bb96d2246 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:07.463219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6dfee-4ff6-4578-a161-3f6bb96d2246 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:07.464153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 202/415] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:07.464639 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a6dfee-4ff6-4578-a161-3f6bb96d2246 None admin] PATH_INFO: `/v3/projects/b206fdda52da4f339c6c008d4c7973af` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:07.472673 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6dfee-4ff6-4578-a161-3f6bb96d2246 None admin] RBAC: Authorizing `identity:get_project(project_id=b206fdda52da4f339c6c008d4c7973af)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:07.474390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a6dfee-4ff6-4578-a161-3f6bb96d2246 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:07.476316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 214/416] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:39:07 2022] GET /identity/v3/projects/b206fdda52da4f339c6c008d4c7973af => generated 344 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:07.483101 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7704f0a-341c-49b3-a82a-07ab30e5790e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:07.621788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7704f0a-341c-49b3-a82a-07ab30e5790e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:07.645639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:07.671693 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d643f8-f7b7-4178-b3f6-a870e6dd2bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:07.672949 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d643f8-f7b7-4178-b3f6-a870e6dd2bef None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:07.673433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d643f8-f7b7-4178-b3f6-a870e6dd2bef None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:07.673901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d643f8-f7b7-4178-b3f6-a870e6dd2bef None admin] PATH_INFO: `/v3/projects/b206fdda52da4f339c6c008d4c7973af/groups/a160a734043a4483988bd33da3922f3e/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:07.681034 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d643f8-f7b7-4178-b3f6-a870e6dd2bef None admin] RBAC: Authorizing `identity:create_grant(project_id=b206fdda52da4f339c6c008d4c7973af, group_id=a160a734043a4483988bd33da3922f3e, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:07.682719 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d643f8-f7b7-4178-b3f6-a870e6dd2bef None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:07.707774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 203/417] 10.222.0.60 () {58 vars in 1442 bytes} [Mon Sep 19 14:39:07 2022] PUT /identity/v3/projects/b206fdda52da4f339c6c008d4c7973af/groups/a160a734043a4483988bd33da3922f3e/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:39:07.816353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6dfee-4ff6-4578-a161-3f6bb96d2246 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:07.821207 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:07.961469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a6dfee-4ff6-4578-a161-3f6bb96d2246 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:08.001733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668e552-a3a5-42a6-b10e-76e443bc324a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:08.002735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668e552-a3a5-42a6-b10e-76e443bc324a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:08.003211 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668e552-a3a5-42a6-b10e-76e443bc324a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:08.003677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668e552-a3a5-42a6-b10e-76e443bc324a None admin] PATH_INFO: `/v3/roles/member` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:08.017413 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668e552-a3a5-42a6-b10e-76e443bc324a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-1668e552-a3a5-42a6-b10e-76e443bc324a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:08.023183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:08.024082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:08.024914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:08.024914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:08.024914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:08.024914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:08.024914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:08.024914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:08.024914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:08.024914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:08.024914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:08.024914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:08.024914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:39:08.024914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:08.027347 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 215/418] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:08.034069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d643f8-f7b7-4178-b3f6-a870e6dd2bef None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:08.195509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d643f8-f7b7-4178-b3f6-a870e6dd2bef None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:08.242676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb21a40-078d-4934-9e26-2fe944b81e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:08.243625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb21a40-078d-4934-9e26-2fe944b81e1f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:08.244135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb21a40-078d-4934-9e26-2fe944b81e1f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:08.244600 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb21a40-078d-4934-9e26-2fe944b81e1f None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:08.245470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb21a40-078d-4934-9e26-2fe944b81e1f None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:08.245960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb21a40-078d-4934-9e26-2fe944b81e1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:08.247637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb21a40-078d-4934-9e26-2fe944b81e1f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:08.260346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 204/419] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:39:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:08.266715 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668e552-a3a5-42a6-b10e-76e443bc324a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:08.348627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:08.407483 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668e552-a3a5-42a6-b10e-76e443bc324a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:08.457815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217059c1-a702-4310-b6c6-d35603771a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:08.458458 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217059c1-a702-4310-b6c6-d35603771a72 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:08.458742 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217059c1-a702-4310-b6c6-d35603771a72 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:08.459018 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217059c1-a702-4310-b6c6-d35603771a72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:08.571366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217059c1-a702-4310-b6c6-d35603771a72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:08.572870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217059c1-a702-4310-b6c6-d35603771a72 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:08.605781 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5844281-6177-428e-b02e-cecf2892becb None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:08.606410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5844281-6177-428e-b02e-cecf2892becb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:08.607050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5844281-6177-428e-b02e-cecf2892becb None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:08.613063 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 205/420] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:08.616603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91041c4a-3411-4bf6-8b92-af22cad9c06e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:08.616889 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91041c4a-3411-4bf6-8b92-af22cad9c06e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:08.617161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91041c4a-3411-4bf6-8b92-af22cad9c06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:08.647394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:08.654423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-91041c4a-3411-4bf6-8b92-af22cad9c06e None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:08.655629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-91041c4a-3411-4bf6-8b92-af22cad9c06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:08.707674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:08.714691 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 216/421] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:08.720672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217059c1-a702-4310-b6c6-d35603771a72 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:08.780428 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 206/422] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:08.888646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217059c1-a702-4310-b6c6-d35603771a72 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:08.937501 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c8d78-ee7a-402f-b57f-9cc172be1ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:08.938132 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c8d78-ee7a-402f-b57f-9cc172be1ce6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:08.938412 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c8d78-ee7a-402f-b57f-9cc172be1ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:08.938687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c8d78-ee7a-402f-b57f-9cc172be1ce6 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:08.944489 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c8d78-ee7a-402f-b57f-9cc172be1ce6 None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:08.945927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c8d78-ee7a-402f-b57f-9cc172be1ce6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:08.947531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 217/423] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:39:08 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:08.953804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91041c4a-3411-4bf6-8b92-af22cad9c06e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:09.081033 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c8d78-ee7a-402f-b57f-9cc172be1ce6 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:09.112051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91041c4a-3411-4bf6-8b92-af22cad9c06e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:09.144012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:09.177943 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d8f01-3ca7-429d-a41d-a30d468a9f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:09.178954 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d8f01-3ca7-429d-a41d-a30d468a9f7e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:09.179457 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d8f01-3ca7-429d-a41d-a30d468a9f7e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:09.179956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d8f01-3ca7-429d-a41d-a30d468a9f7e None admin] PATH_INFO: `/v3/groups/2620063028584525a9ef3d5e557d96b6` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:09.182073 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d8f01-3ca7-429d-a41d-a30d468a9f7e None admin] RBAC: Authorizing `identity:get_group(group_id=2620063028584525a9ef3d5e557d96b6)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:09.183746 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d8f01-3ca7-429d-a41d-a30d468a9f7e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:09.189313 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 207/424] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Sep 19 14:39:08 2022] GET /identity/v3/groups/2620063028584525a9ef3d5e557d96b6 => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:09.201670 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d8f01-3ca7-429d-a41d-a30d468a9f7e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:09.250814 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c8d78-ee7a-402f-b57f-9cc172be1ce6 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:09.267150 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:09.288692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ec848-d49d-4902-a3e1-8a70e5792c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:09.289330 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ec848-d49d-4902-a3e1-8a70e5792c9c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:09.289609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ec848-d49d-4902-a3e1-8a70e5792c9c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:09.289877 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ec848-d49d-4902-a3e1-8a70e5792c9c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:09.301553 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ec848-d49d-4902-a3e1-8a70e5792c9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-c52ec848-d49d-4902-a3e1-8a70e5792c9c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:09.306367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:09.307273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:09.308108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:09.308108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:09.308108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:09.308108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:09.308108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:09.308108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:09.308108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:09.308108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:09.308108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:09.308108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:09.308108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:39:09.308108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:09.308108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 218/425] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:39:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:39:09.313837 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ec848-d49d-4902-a3e1-8a70e5792c9c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:09.378018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d8f01-3ca7-429d-a41d-a30d468a9f7e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:09.444848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfbca3-4e07-46f7-b513-2d923c456474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:09.445453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfbca3-4e07-46f7-b513-2d923c456474 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:09.445737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfbca3-4e07-46f7-b513-2d923c456474 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:09.446003 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfbca3-4e07-46f7-b513-2d923c456474 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:09.446725 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfbca3-4e07-46f7-b513-2d923c456474 None admin] RBAC: Adding query filter params (group.id=2620063028584525a9ef3d5e557d96b6, role.id=0a6592b9a0b64df6938aba0a010c2256, scope.project.id=4681da4f1deb408293b2c948aec851ea) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:09.447027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfbca3-4e07-46f7-b513-2d923c456474 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:09.448491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfbca3-4e07-46f7-b513-2d923c456474 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:09.465717 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 208/426] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Sep 19 14:39:09 2022] GET /identity/v3/role_assignments?group.id=2620063028584525a9ef3d5e557d96b6&scope.project.id=4681da4f1deb408293b2c948aec851ea&role.id=0a6592b9a0b64df6938aba0a010c2256 => generated 611 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:09.490994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:09.502888 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ec848-d49d-4902-a3e1-8a70e5792c9c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:09.544506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96f217-5641-4757-822e-e7ff7d659f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:09.545130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96f217-5641-4757-822e-e7ff7d659f98 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:09.545407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96f217-5641-4757-822e-e7ff7d659f98 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:09.545682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96f217-5641-4757-822e-e7ff7d659f98 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:09.546353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96f217-5641-4757-822e-e7ff7d659f98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:09.546655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96f217-5641-4757-822e-e7ff7d659f98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:09.548325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96f217-5641-4757-822e-e7ff7d659f98 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:09.557679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 219/427] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:39:09.565005 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfbca3-4e07-46f7-b513-2d923c456474 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:09.709174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:09.712584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfbca3-4e07-46f7-b513-2d923c456474 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:09.754722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07000e63-3c6b-458a-9992-0a3d2ce8bfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:09.755355 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07000e63-3c6b-458a-9992-0a3d2ce8bfdc None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:09.755644 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07000e63-3c6b-458a-9992-0a3d2ce8bfdc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:09.755965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07000e63-3c6b-458a-9992-0a3d2ce8bfdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:09.772732 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8abdb4-3791-4ae4-9f90-26bb240967e2 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:09.773025 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8abdb4-3791-4ae4-9f90-26bb240967e2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:09.773291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8abdb4-3791-4ae4-9f90-26bb240967e2 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:09.775096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 220/428] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:09.780817 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f7c7d-a84a-46bc-a9ee-7ef6bbe1f44f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:09.781340 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f7c7d-a84a-46bc-a9ee-7ef6bbe1f44f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:09.781802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f7c7d-a84a-46bc-a9ee-7ef6bbe1f44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:09.802321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5c4f7c7d-a84a-46bc-a9ee-7ef6bbe1f44f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:09.804062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5c4f7c7d-a84a-46bc-a9ee-7ef6bbe1f44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:09.850054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07000e63-3c6b-458a-9992-0a3d2ce8bfdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:09.853257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07000e63-3c6b-458a-9992-0a3d2ce8bfdc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:09.901475 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 221/429] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:39:09.902784 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e02d7-5ef0-44dd-89a6-a8ed0efffa98 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:09.903062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e02d7-5ef0-44dd-89a6-a8ed0efffa98 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:09.904004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e02d7-5ef0-44dd-89a6-a8ed0efffa98 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:09.905538 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 222/430] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:09.910820 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b296a-0fd7-4245-9af0-04d732446809 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:09.911101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b296a-0fd7-4245-9af0-04d732446809 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:09.911377 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b296a-0fd7-4245-9af0-04d732446809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:09.936131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-508b296a-0fd7-4245-9af0-04d732446809 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:09.937489 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-508b296a-0fd7-4245-9af0-04d732446809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:09.987091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:09.991226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 209/431] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:09.996640 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07000e63-3c6b-458a-9992-0a3d2ce8bfdc None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:10.022206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 223/432] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:39:10.104788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07000e63-3c6b-458a-9992-0a3d2ce8bfdc None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:10.144368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61688dd8-9fd7-48e0-b8f1-8ddacf2ecb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:10.145103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61688dd8-9fd7-48e0-b8f1-8ddacf2ecb95 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:10.145287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61688dd8-9fd7-48e0-b8f1-8ddacf2ecb95 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:10.145550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61688dd8-9fd7-48e0-b8f1-8ddacf2ecb95 None admin] PATH_INFO: `/v3/projects/4681da4f1deb408293b2c948aec851ea` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:10.146734 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61688dd8-9fd7-48e0-b8f1-8ddacf2ecb95 None admin] RBAC: Authorizing `identity:get_project(project_id=4681da4f1deb408293b2c948aec851ea)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:10.148240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61688dd8-9fd7-48e0-b8f1-8ddacf2ecb95 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:10.150167 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 210/433] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:39:09 2022] GET /identity/v3/projects/4681da4f1deb408293b2c948aec851ea => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:10.155998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b296a-0fd7-4245-9af0-04d732446809 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:10.181289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:10.234295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61688dd8-9fd7-48e0-b8f1-8ddacf2ecb95 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:10.286655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b296a-0fd7-4245-9af0-04d732446809 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:10.321168 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:10.328684 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158c6a3-0f0c-437f-a5c5-55e4e3f850b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:10.329455 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158c6a3-0f0c-437f-a5c5-55e4e3f850b9 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:10.330243 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158c6a3-0f0c-437f-a5c5-55e4e3f850b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:10.330243 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158c6a3-0f0c-437f-a5c5-55e4e3f850b9 None admin] PATH_INFO: `/v3/groups/2620063028584525a9ef3d5e557d96b6` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:10.331305 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158c6a3-0f0c-437f-a5c5-55e4e3f850b9 None admin] RBAC: Authorizing `identity:get_group(group_id=2620063028584525a9ef3d5e557d96b6)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:10.332958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158c6a3-0f0c-437f-a5c5-55e4e3f850b9 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:10.340476 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 224/434] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Sep 19 14:39:10 2022] GET /identity/v3/groups/2620063028584525a9ef3d5e557d96b6 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:10.341722 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158c6a3-0f0c-437f-a5c5-55e4e3f850b9 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:10.374374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61688dd8-9fd7-48e0-b8f1-8ddacf2ecb95 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:10.415721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3536dc-0e10-4f68-b07a-6b290ffa5268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:10.416472 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3536dc-0e10-4f68-b07a-6b290ffa5268 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:10.416757 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3536dc-0e10-4f68-b07a-6b290ffa5268 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:10.417030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3536dc-0e10-4f68-b07a-6b290ffa5268 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:10.437371 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3536dc-0e10-4f68-b07a-6b290ffa5268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:10.440055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-5b3536dc-0e10-4f68-b07a-6b290ffa5268 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:10.442443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:10.443357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:10.444211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:10.444211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:10.444211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:10.444211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:10.444211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:10.444211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:10.444211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:10.444211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:10.444211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:10.444211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:10.444211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:39:10.444211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:10.447983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 211/435] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:39:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:10.448850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3536dc-0e10-4f68-b07a-6b290ffa5268 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:10.463849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158c6a3-0f0c-437f-a5c5-55e4e3f850b9 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:10.497853 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5728e6-e358-4d71-a063-cfb07568bfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:10.498569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5728e6-e358-4d71-a063-cfb07568bfd4 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:10.498856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5728e6-e358-4d71-a063-cfb07568bfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:10.499135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5728e6-e358-4d71-a063-cfb07568bfd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:10.499887 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5728e6-e358-4d71-a063-cfb07568bfd4 None admin] RBAC: Adding query filter params (group.id=2620063028584525a9ef3d5e557d96b6, role.id=fe3bbad209634f16b1fcb66d6d49f665, scope.project.id=4681da4f1deb408293b2c948aec851ea) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:10.500230 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5728e6-e358-4d71-a063-cfb07568bfd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:10.501660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5728e6-e358-4d71-a063-cfb07568bfd4 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:10.507612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 225/436] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Sep 19 14:39:10 2022] GET /identity/v3/role_assignments?group.id=2620063028584525a9ef3d5e557d96b6&scope.project.id=4681da4f1deb408293b2c948aec851ea&role.id=fe3bbad209634f16b1fcb66d6d49f665 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:10.508947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5728e6-e358-4d71-a063-cfb07568bfd4 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:10.555814 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3536dc-0e10-4f68-b07a-6b290ffa5268 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:10.585292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:10.594556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d1e7a-0f9d-4d0f-91b0-0a0837057fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:10.595212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d1e7a-0f9d-4d0f-91b0-0a0837057fd4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:10.595491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d1e7a-0f9d-4d0f-91b0-0a0837057fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:10.595763 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d1e7a-0f9d-4d0f-91b0-0a0837057fd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:10.608762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d1e7a-0f9d-4d0f-91b0-0a0837057fd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-362d1e7a-0f9d-4d0f-91b0-0a0837057fd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:10.613788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:10.614938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:10.616997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:10.616997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:10.616997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:10.616997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:10.616997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:10.616997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:10.616997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:10.616997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:10.616997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:10.616997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:10.616997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 19 14:39:10.616997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:10.616997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 212/437] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:10.616997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867abf6-9d95-4b94-b8e6-5af5fdc4b193 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:10.617832 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867abf6-9d95-4b94-b8e6-5af5fdc4b193 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:10.617832 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867abf6-9d95-4b94-b8e6-5af5fdc4b193 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:10.623458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 213/438] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:10.623458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867abf6-9d95-4b94-b8e6-5af5fdc4b193 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:10.642654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5728e6-e358-4d71-a063-cfb07568bfd4 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:10.683480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4cbc4-2053-4a58-a5b3-1c687187c91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:10.684127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4cbc4-2053-4a58-a5b3-1c687187c91e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:10.684408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4cbc4-2053-4a58-a5b3-1c687187c91e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:10.685209 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4cbc4-2053-4a58-a5b3-1c687187c91e None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:10.685874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4cbc4-2053-4a58-a5b3-1c687187c91e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:10.686179 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4cbc4-2053-4a58-a5b3-1c687187c91e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:10.687547 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4cbc4-2053-4a58-a5b3-1c687187c91e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:10.698043 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 226/439] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:10.700003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5cf77-eb62-41c1-9e8f-0be61faae6f9 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:10.700669 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5cf77-eb62-41c1-9e8f-0be61faae6f9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:10.701120 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa5cf77-eb62-41c1-9e8f-0be61faae6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:10.724681 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4aa5cf77-eb62-41c1-9e8f-0be61faae6f9 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:10.725869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4aa5cf77-eb62-41c1-9e8f-0be61faae6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:10.727960 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:10.745792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867abf6-9d95-4b94-b8e6-5af5fdc4b193 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:10.786388 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3274b85-1b56-4595-8d13-25b384c54eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:10.787004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3274b85-1b56-4595-8d13-25b384c54eb5 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:10.787287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3274b85-1b56-4595-8d13-25b384c54eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:10.787560 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3274b85-1b56-4595-8d13-25b384c54eb5 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:10.788265 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3274b85-1b56-4595-8d13-25b384c54eb5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:10.789469 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3274b85-1b56-4595-8d13-25b384c54eb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:10.789959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3274b85-1b56-4595-8d13-25b384c54eb5 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:10.801928 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 214/440] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:39:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:10.802702 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3274b85-1b56-4595-8d13-25b384c54eb5 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:10.803359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 227/441] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:10.809359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5cf77-eb62-41c1-9e8f-0be61faae6f9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:10.887635 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:10.957924 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa5cf77-eb62-41c1-9e8f-0be61faae6f9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:10.958453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3274b85-1b56-4595-8d13-25b384c54eb5 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:10.995770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be500192-ec85-44bf-933b-fde374bbadc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:10.996422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be500192-ec85-44bf-933b-fde374bbadc9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:10.996688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be500192-ec85-44bf-933b-fde374bbadc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:10.996963 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be500192-ec85-44bf-933b-fde374bbadc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:10.997558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574e355-694f-45d7-add6-56c8defd34f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:10.998163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574e355-694f-45d7-add6-56c8defd34f2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:10.998440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574e355-694f-45d7-add6-56c8defd34f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:10.998709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574e355-694f-45d7-add6-56c8defd34f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:11.007446 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:11.068719 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be500192-ec85-44bf-933b-fde374bbadc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:11.070151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be500192-ec85-44bf-933b-fde374bbadc9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:11.077002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574e355-694f-45d7-add6-56c8defd34f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:11.078706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574e355-694f-45d7-add6-56c8defd34f2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:11.126445 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:11.183042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 215/442] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:11.183936 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be500192-ec85-44bf-933b-fde374bbadc9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:11.192513 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 228/443] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 14:39:11.193430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574e355-694f-45d7-add6-56c8defd34f2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:11.231054 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:11.281727 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be500192-ec85-44bf-933b-fde374bbadc9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:11.290968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574e355-694f-45d7-add6-56c8defd34f2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:11.315754 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fae96-a6ae-4cf4-9a32-679aa6f2cdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:11.316408 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fae96-a6ae-4cf4-9a32-679aa6f2cdaf None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:11.316690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fae96-a6ae-4cf4-9a32-679aa6f2cdaf None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:11.316968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fae96-a6ae-4cf4-9a32-679aa6f2cdaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:11.325375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58bdc2-2e8a-4f90-be34-1071c3bbcf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:11.325941 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58bdc2-2e8a-4f90-be34-1071c3bbcf4f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:11.326219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58bdc2-2e8a-4f90-be34-1071c3bbcf4f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:11.326489 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58bdc2-2e8a-4f90-be34-1071c3bbcf4f None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:11.327404 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fae96-a6ae-4cf4-9a32-679aa6f2cdaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:11.327598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c58bdc2-2e8a-4f90-be34-1071c3bbcf4f None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:11.329081 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c58bdc2-2e8a-4f90-be34-1071c3bbcf4f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:11.330700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 229/444] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:39:11 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-961fae96-a6ae-4cf4-9a32-679aa6f2cdaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:11.332609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:11.333492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:11.334323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:11.334323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:11.334323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:11.334323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:11.334323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:11.334323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:11.334323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:11.334323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:11.334323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:11.334323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:11.334323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:11.334323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:11.334939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58bdc2-2e8a-4f90-be34-1071c3bbcf4f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:11.335016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 216/445] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:39:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:11.336554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fae96-a6ae-4cf4-9a32-679aa6f2cdaf None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:11.348800 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:11.433644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58bdc2-2e8a-4f90-be34-1071c3bbcf4f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:11.437127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fae96-a6ae-4cf4-9a32-679aa6f2cdaf None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:11.441089 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:11.472308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b36b27-fa66-42c4-aa38-1f06e54e8420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:11.472937 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b36b27-fa66-42c4-aa38-1f06e54e8420 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:11.473210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b36b27-fa66-42c4-aa38-1f06e54e8420 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:11.473479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b36b27-fa66-42c4-aa38-1f06e54e8420 None admin] PATH_INFO: `/v3/groups/2620063028584525a9ef3d5e557d96b6` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:11.473599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610058e-b30f-4179-acfc-28d68a31cd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:11.474235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610058e-b30f-4179-acfc-28d68a31cd28 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:11.474517 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610058e-b30f-4179-acfc-28d68a31cd28 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:11.474609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b36b27-fa66-42c4-aa38-1f06e54e8420 None admin] RBAC: Authorizing `identity:get_group(group_id=2620063028584525a9ef3d5e557d96b6)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:11.474795 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610058e-b30f-4179-acfc-28d68a31cd28 None admin] PATH_INFO: `/v3/projects/17924018ca584f3696136b838ab2fdc8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:11.475938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610058e-b30f-4179-acfc-28d68a31cd28 None admin] RBAC: Authorizing `identity:get_project(project_id=17924018ca584f3696136b838ab2fdc8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:11.476054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b36b27-fa66-42c4-aa38-1f06e54e8420 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:11.477357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610058e-b30f-4179-acfc-28d68a31cd28 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:11.477594 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 217/446] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Sep 19 14:39:11 2022] GET /identity/v3/groups/2620063028584525a9ef3d5e557d96b6 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:39:11.478336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b36b27-fa66-42c4-aa38-1f06e54e8420 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:11.483897 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 230/447] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:39:11 2022] GET /identity/v3/projects/17924018ca584f3696136b838ab2fdc8 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:11.484037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610058e-b30f-4179-acfc-28d68a31cd28 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:11.560180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:11.582056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b36b27-fa66-42c4-aa38-1f06e54e8420 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:11.582735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610058e-b30f-4179-acfc-28d68a31cd28 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:11.617989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdac609-1922-4c85-955d-dc3ba19921ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:11.618494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8ffb4-8c08-47ac-9743-286359a9491e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:11.618616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdac609-1922-4c85-955d-dc3ba19921ae None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:11.618858 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdac609-1922-4c85-955d-dc3ba19921ae None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:11.619097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8ffb4-8c08-47ac-9743-286359a9491e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:11.619199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdac609-1922-4c85-955d-dc3ba19921ae None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:11.619380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8ffb4-8c08-47ac-9743-286359a9491e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:11.619658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8ffb4-8c08-47ac-9743-286359a9491e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:11.619808 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdac609-1922-4c85-955d-dc3ba19921ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:11.620153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdac609-1922-4c85-955d-dc3ba19921ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:11.620409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8ffb4-8c08-47ac-9743-286359a9491e None admin] RBAC: Adding query filter params (group.id=2620063028584525a9ef3d5e557d96b6, role.id=fe3bbad209634f16b1fcb66d6d49f665, scope.project.id=17924018ca584f3696136b838ab2fdc8) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:11.620720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8ffb4-8c08-47ac-9743-286359a9491e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:11.621578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdac609-1922-4c85-955d-dc3ba19921ae None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:11.622202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8ffb4-8c08-47ac-9743-286359a9491e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:11.627921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 231/448] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Sep 19 14:39:11 2022] GET /identity/v3/role_assignments?group.id=2620063028584525a9ef3d5e557d96b6&scope.project.id=17924018ca584f3696136b838ab2fdc8&role.id=fe3bbad209634f16b1fcb66d6d49f665 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:39:11.628709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8ffb4-8c08-47ac-9743-286359a9491e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:11.630333 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 218/449] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:11.645644 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdac609-1922-4c85-955d-dc3ba19921ae None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:11.673144 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:11.743229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8ffb4-8c08-47ac-9743-286359a9491e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:11.761676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdac609-1922-4c85-955d-dc3ba19921ae None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:11.783331 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282baf8-516b-40ac-8b9e-53359cf434f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:11.784017 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282baf8-516b-40ac-8b9e-53359cf434f5 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:11.784297 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282baf8-516b-40ac-8b9e-53359cf434f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:11.784567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282baf8-516b-40ac-8b9e-53359cf434f5 None admin] PATH_INFO: `/v3/groups/2620063028584525a9ef3d5e557d96b6` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:11.785749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282baf8-516b-40ac-8b9e-53359cf434f5 None admin] RBAC: Authorizing `identity:get_group(group_id=2620063028584525a9ef3d5e557d96b6)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:11.787225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282baf8-516b-40ac-8b9e-53359cf434f5 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:11.793680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 232/450] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Sep 19 14:39:11 2022] GET /identity/v3/groups/2620063028584525a9ef3d5e557d96b6 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:11.794874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282baf8-516b-40ac-8b9e-53359cf434f5 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:11.806826 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95e0c9-8882-4441-a107-9a8ecb4c35c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:11.807454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95e0c9-8882-4441-a107-9a8ecb4c35c6 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:11.808084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95e0c9-8882-4441-a107-9a8ecb4c35c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:11.808365 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95e0c9-8882-4441-a107-9a8ecb4c35c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:11.824853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:11.896679 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95e0c9-8882-4441-a107-9a8ecb4c35c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:11.898147 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95e0c9-8882-4441-a107-9a8ecb4c35c6 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:11.908335 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282baf8-516b-40ac-8b9e-53359cf434f5 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:11.915465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:11.943117 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba602a2-7a15-4698-a8d9-05aa468e737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:11.943610 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba602a2-7a15-4698-a8d9-05aa468e737c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:11.943929 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba602a2-7a15-4698-a8d9-05aa468e737c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:11.944202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba602a2-7a15-4698-a8d9-05aa468e737c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:11.944916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba602a2-7a15-4698-a8d9-05aa468e737c None admin] RBAC: Adding query filter params (group.id=2620063028584525a9ef3d5e557d96b6, role.id=0a6592b9a0b64df6938aba0a010c2256, scope.project.id=17924018ca584f3696136b838ab2fdc8) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:11.945317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba602a2-7a15-4698-a8d9-05aa468e737c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:11.947437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba602a2-7a15-4698-a8d9-05aa468e737c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:11.954457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 233/451] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Sep 19 14:39:11 2022] GET /identity/v3/role_assignments?group.id=2620063028584525a9ef3d5e557d96b6&scope.project.id=17924018ca584f3696136b838ab2fdc8&role.id=0a6592b9a0b64df6938aba0a010c2256 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:39:12.003534 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 219/452] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:12.009730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba602a2-7a15-4698-a8d9-05aa468e737c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:12.120005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba602a2-7a15-4698-a8d9-05aa468e737c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:12.155608 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b4ebd-fb34-4b58-aad0-cc895e366138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:12.156239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b4ebd-fb34-4b58-aad0-cc895e366138 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:12.156525 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b4ebd-fb34-4b58-aad0-cc895e366138 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:12.156799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b4ebd-fb34-4b58-aad0-cc895e366138 None admin] PATH_INFO: `/v3/projects/b206fdda52da4f339c6c008d4c7973af` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:12.157933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b4ebd-fb34-4b58-aad0-cc895e366138 None admin] RBAC: Authorizing `identity:get_project(project_id=b206fdda52da4f339c6c008d4c7973af)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:12.159375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198b4ebd-fb34-4b58-aad0-cc895e366138 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:12.161024 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 234/453] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:39:12 2022] GET /identity/v3/projects/b206fdda52da4f339c6c008d4c7973af => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:12.166589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95e0c9-8882-4441-a107-9a8ecb4c35c6 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:12.210741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:12.273234 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95e0c9-8882-4441-a107-9a8ecb4c35c6 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:12.307884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4420ab-d402-4d16-b4ef-a234c8b8cac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:12.308509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4420ab-d402-4d16-b4ef-a234c8b8cac6 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:12.308785 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4420ab-d402-4d16-b4ef-a234c8b8cac6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:12.309054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4420ab-d402-4d16-b4ef-a234c8b8cac6 None admin] PATH_INFO: `/v3/groups/a160a734043a4483988bd33da3922f3e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:12.310154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4420ab-d402-4d16-b4ef-a234c8b8cac6 None admin] RBAC: Authorizing `identity:get_group(group_id=a160a734043a4483988bd33da3922f3e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:12.311595 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4420ab-d402-4d16-b4ef-a234c8b8cac6 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:12.313479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 220/454] 10.222.0.60 () {56 vars in 1180 bytes} [Mon Sep 19 14:39:12 2022] GET /identity/v3/groups/a160a734043a4483988bd33da3922f3e => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:12.319248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b4ebd-fb34-4b58-aad0-cc895e366138 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:12.362812 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:12.425233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b4ebd-fb34-4b58-aad0-cc895e366138 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:12.459428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05850fe2-0098-4730-8eb7-b0c06fcaaaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:12.460078 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05850fe2-0098-4730-8eb7-b0c06fcaaaf6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:12.460364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05850fe2-0098-4730-8eb7-b0c06fcaaaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:12.460636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05850fe2-0098-4730-8eb7-b0c06fcaaaf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:12.461366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05850fe2-0098-4730-8eb7-b0c06fcaaaf6 None admin] RBAC: Adding query filter params (group.id=a160a734043a4483988bd33da3922f3e, role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=b206fdda52da4f339c6c008d4c7973af) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:12.461671 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05850fe2-0098-4730-8eb7-b0c06fcaaaf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:12.463095 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05850fe2-0098-4730-8eb7-b0c06fcaaaf6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:12.469561 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 235/455] 10.222.0.60 () {56 vars in 1376 bytes} [Mon Sep 19 14:39:12 2022] GET /identity/v3/role_assignments?group.id=a160a734043a4483988bd33da3922f3e&scope.project.id=b206fdda52da4f339c6c008d4c7973af&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:15.048945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9148b91-eee5-48bd-9bb7-b56b98a9cff9 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:15.049859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9148b91-eee5-48bd-9bb7-b56b98a9cff9 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:15.050279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9148b91-eee5-48bd-9bb7-b56b98a9cff9 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:15.052169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 221/456] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:15.057876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5f96e-c801-4157-8365-5d8a2f28cb70 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:15.058713 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5f96e-c801-4157-8365-5d8a2f28cb70 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:15.059159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5f96e-c801-4157-8365-5d8a2f28cb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:15.085407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9fc5f96e-c801-4157-8365-5d8a2f28cb70 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:15.086833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9fc5f96e-c801-4157-8365-5d8a2f28cb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:15.159562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b2481-4025-49d5-a04e-9f531bd271cb None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:15.159944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b2481-4025-49d5-a04e-9f531bd271cb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:15.160141 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b2481-4025-49d5-a04e-9f531bd271cb None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:15.161610 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 222/457] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:39:15.166984 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008dea2-80ab-4d52-9fdd-dd7b3f371cde None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:15.167259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008dea2-80ab-4d52-9fdd-dd7b3f371cde None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:15.167534 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008dea2-80ab-4d52-9fdd-dd7b3f371cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:15.193131 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0008dea2-80ab-4d52-9fdd-dd7b3f371cde None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:15.194331 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0008dea2-80ab-4d52-9fdd-dd7b3f371cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:15.209385 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 236/458] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:15.299183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 223/459] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:39:15.396500 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b774204-a3d5-4420-b948-bb67145085f7 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:15.396820 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b774204-a3d5-4420-b948-bb67145085f7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:15.397058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b774204-a3d5-4420-b948-bb67145085f7 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:15.398646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 237/460] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:15.404700 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f5e25-3d4e-4135-b269-41f1be3db955 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:15.404990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f5e25-3d4e-4135-b269-41f1be3db955 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:15.405266 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f5e25-3d4e-4135-b269-41f1be3db955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:15.434140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-174f5e25-3d4e-4135-b269-41f1be3db955 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:15.436198 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-174f5e25-3d4e-4135-b269-41f1be3db955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:15.478696 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:15.512750 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b774204-a3d5-4420-b948-bb67145085f7 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:15.517963 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 224/461] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:15.546474 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:15.641562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b774204-a3d5-4420-b948-bb67145085f7 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:15.683289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89463182-2a54-49b2-89fc-30a35726597b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:15.683964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89463182-2a54-49b2-89fc-30a35726597b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:15.684249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89463182-2a54-49b2-89fc-30a35726597b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:15.684524 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89463182-2a54-49b2-89fc-30a35726597b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:15.690509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f5e25-3d4e-4135-b269-41f1be3db955 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:15.695451 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89463182-2a54-49b2-89fc-30a35726597b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:15.696957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89463182-2a54-49b2-89fc-30a35726597b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-89463182-2a54-49b2-89fc-30a35726597b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:15.720916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:15.721891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:15.722749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:15.722749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:15.722749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:15.731982 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 238/462] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:15.742601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89463182-2a54-49b2-89fc-30a35726597b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:15.835303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f5e25-3d4e-4135-b269-41f1be3db955 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:15.879462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc2e40-5d85-4c12-a54f-82f672ef0bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:15.880159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc2e40-5d85-4c12-a54f-82f672ef0bd4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:15.880539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc2e40-5d85-4c12-a54f-82f672ef0bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:15.880817 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc2e40-5d85-4c12-a54f-82f672ef0bd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:15.884918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:15.895376 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc2e40-5d85-4c12-a54f-82f672ef0bd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:15.896930 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc2e40-5d85-4c12-a54f-82f672ef0bd4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:15.908057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89463182-2a54-49b2-89fc-30a35726597b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-b8fc2e40-5d85-4c12-a54f-82f672ef0bd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:15.913535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:15.914453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:15.915338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:15.915338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:15.915338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:15.919926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 225/463] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:39:15.920980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc2e40-5d85-4c12-a54f-82f672ef0bd4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:15.946181 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:15.951495 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3163c8-3f78-49f6-b0bc-33c6451017cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:15.952796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3163c8-3f78-49f6-b0bc-33c6451017cb None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:15.953279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3163c8-3f78-49f6-b0bc-33c6451017cb None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:15.953743 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3163c8-3f78-49f6-b0bc-33c6451017cb None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:15.954575 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3163c8-3f78-49f6-b0bc-33c6451017cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:15.955087 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3163c8-3f78-49f6-b0bc-33c6451017cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:15.956888 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3163c8-3f78-49f6-b0bc-33c6451017cb None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:15.975182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 239/464] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:15.976927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3163c8-3f78-49f6-b0bc-33c6451017cb None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:16.050275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc2e40-5d85-4c12-a54f-82f672ef0bd4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:16.091570 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d8fb0-ea99-499b-84d4-bbf37789dd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:16.092684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d8fb0-ea99-499b-84d4-bbf37789dd57 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:16.092684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d8fb0-ea99-499b-84d4-bbf37789dd57 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:16.092891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d8fb0-ea99-499b-84d4-bbf37789dd57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:16.104822 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d8fb0-ea99-499b-84d4-bbf37789dd57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:16.105999 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3163c8-3f78-49f6-b0bc-33c6451017cb None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:16.106407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d8fb0-ea99-499b-84d4-bbf37789dd57 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-685d8fb0-ea99-499b-84d4-bbf37789dd57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:16.123547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:16.124625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:16.125707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:16.125707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:16.125707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:16.125707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 226/465] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:16.126395 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d8fb0-ea99-499b-84d4-bbf37789dd57 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:16.152968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36bf33-a45c-4045-9cd8-a8bf831e9e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:16.154175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36bf33-a45c-4045-9cd8-a8bf831e9e5b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:16.154457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36bf33-a45c-4045-9cd8-a8bf831e9e5b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:16.155003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36bf33-a45c-4045-9cd8-a8bf831e9e5b None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:16.155648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36bf33-a45c-4045-9cd8-a8bf831e9e5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:16.156084 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36bf33-a45c-4045-9cd8-a8bf831e9e5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:16.157560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36bf33-a45c-4045-9cd8-a8bf831e9e5b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:16.171245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 240/466] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:16.172185 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36bf33-a45c-4045-9cd8-a8bf831e9e5b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:16.176343 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:16.210064 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:16.251312 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d8fb0-ea99-499b-84d4-bbf37789dd57 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:16.289208 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b413dd-11b8-44e1-a834-a06af7731791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:16.290345 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b413dd-11b8-44e1-a834-a06af7731791 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:16.290778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b413dd-11b8-44e1-a834-a06af7731791 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:16.291603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b413dd-11b8-44e1-a834-a06af7731791 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:16.292445 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b413dd-11b8-44e1-a834-a06af7731791 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:16.295148 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b413dd-11b8-44e1-a834-a06af7731791 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:16.307864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36bf33-a45c-4045-9cd8-a8bf831e9e5b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:16.327378 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 227/467] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:39:16 2022] POST /identity/v3/users => generated 246 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:39:16.328303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b413dd-11b8-44e1-a834-a06af7731791 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:16.360805 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e5043-8637-4d5b-853d-1803a1b8db80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:16.361934 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e5043-8637-4d5b-853d-1803a1b8db80 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:16.362424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e5043-8637-4d5b-853d-1803a1b8db80 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:16.362933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e5043-8637-4d5b-853d-1803a1b8db80 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:16.364227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e5043-8637-4d5b-853d-1803a1b8db80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:16.365252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e5043-8637-4d5b-853d-1803a1b8db80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:16.367187 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e5043-8637-4d5b-853d-1803a1b8db80 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:16.383173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 241/468] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:16.385266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4c2b5-264e-4167-9385-2d8338c2ea3f None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:16.385808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4c2b5-264e-4167-9385-2d8338c2ea3f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:16.386386 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4c2b5-264e-4167-9385-2d8338c2ea3f None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:16.393180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 242/469] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:16.395319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4c2b5-264e-4167-9385-2d8338c2ea3f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:16.448591 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b413dd-11b8-44e1-a834-a06af7731791 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:16.480784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:16.492088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:16.494127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541226c-887a-403d-bba2-c871db849f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:16.494795 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541226c-887a-403d-bba2-c871db849f1e None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:16.495060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541226c-887a-403d-bba2-c871db849f1e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:16.495417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541226c-887a-403d-bba2-c871db849f1e None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:16.496260 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541226c-887a-403d-bba2-c871db849f1e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:16.497847 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d541226c-887a-403d-bba2-c871db849f1e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:16.522483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 228/470] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:39:16 2022] POST /identity/v3/users => generated 248 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:39:16.523733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80a6a5-e921-4af2-b82a-3f0be1a62f95 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:16.524144 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80a6a5-e921-4af2-b82a-3f0be1a62f95 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:16.524383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80a6a5-e921-4af2-b82a-3f0be1a62f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:16.545119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-db80a6a5-e921-4af2-b82a-3f0be1a62f95 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:16.546388 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-db80a6a5-e921-4af2-b82a-3f0be1a62f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:16.562741 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4c2b5-264e-4167-9385-2d8338c2ea3f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:16.611661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde045c-e1f8-43a6-9e54-2ef979b3caf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:16.612660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde045c-e1f8-43a6-9e54-2ef979b3caf9 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:16.613158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde045c-e1f8-43a6-9e54-2ef979b3caf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:16.614285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde045c-e1f8-43a6-9e54-2ef979b3caf9 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:16.615460 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde045c-e1f8-43a6-9e54-2ef979b3caf9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:16.617395 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde045c-e1f8-43a6-9e54-2ef979b3caf9 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:16.630832 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 229/471] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:16.643775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 243/472] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:39:16 2022] POST /identity/v3/users => generated 248 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:39:16.888036 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5e0e2-d727-489f-a3ad-cc9784baf091 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:16.888372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5e0e2-d727-489f-a3ad-cc9784baf091 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:16.888547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5e0e2-d727-489f-a3ad-cc9784baf091 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:16.889996 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 230/473] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:16.895477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3a88d-4531-49ea-9501-3c046727901a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:16.895767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3a88d-4531-49ea-9501-3c046727901a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:16.896065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3a88d-4531-49ea-9501-3c046727901a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:16.915445 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c0b3a88d-4531-49ea-9501-3c046727901a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:16.916989 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c0b3a88d-4531-49ea-9501-3c046727901a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:16.944624 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5e0e2-d727-489f-a3ad-cc9784baf091 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:17.010513 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 244/474] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:17.105440 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5e0e2-d727-489f-a3ad-cc9784baf091 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:17.122476 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:17.154531 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da985b93-16a6-4a42-811d-534a974745f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:17.155180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da985b93-16a6-4a42-811d-534a974745f4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:17.155456 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da985b93-16a6-4a42-811d-534a974745f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:17.155726 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da985b93-16a6-4a42-811d-534a974745f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:17.169518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da985b93-16a6-4a42-811d-534a974745f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:17.170998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da985b93-16a6-4a42-811d-534a974745f4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-da985b93-16a6-4a42-811d-534a974745f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:17.187187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:17.188159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:17.189462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:17.189462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:17.189462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:17.189462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 231/475] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:17.195886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3a88d-4531-49ea-9501-3c046727901a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:17.272772 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:17.333272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3a88d-4531-49ea-9501-3c046727901a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:17.375324 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dda5eb-79e8-49bb-bda3-9891df5bd1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:17.376266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dda5eb-79e8-49bb-bda3-9891df5bd1aa None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:17.376726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dda5eb-79e8-49bb-bda3-9891df5bd1aa None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:17.377172 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dda5eb-79e8-49bb-bda3-9891df5bd1aa None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:17.378019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dda5eb-79e8-49bb-bda3-9891df5bd1aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:17.378495 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dda5eb-79e8-49bb-bda3-9891df5bd1aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:17.380097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dda5eb-79e8-49bb-bda3-9891df5bd1aa None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:17.396776 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 245/476] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:17.402641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da985b93-16a6-4a42-811d-534a974745f4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:17.476694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dda5eb-79e8-49bb-bda3-9891df5bd1aa None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:17.613196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da985b93-16a6-4a42-811d-534a974745f4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:17.642368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dda5eb-79e8-49bb-bda3-9891df5bd1aa None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:17.662428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:17.684557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03269e56-ff1d-4d73-8033-83513469b72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:17.685216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03269e56-ff1d-4d73-8033-83513469b72a None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:17.685497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03269e56-ff1d-4d73-8033-83513469b72a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:17.685771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03269e56-ff1d-4d73-8033-83513469b72a None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:17.686428 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03269e56-ff1d-4d73-8033-83513469b72a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:17.687886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03269e56-ff1d-4d73-8033-83513469b72a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:17.708514 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36325014-5d55-43a5-ba3d-30e4a943c77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:17.709130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325014-5d55-43a5-ba3d-30e4a943c77d None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:17.709412 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325014-5d55-43a5-ba3d-30e4a943c77d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:17.709682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325014-5d55-43a5-ba3d-30e4a943c77d None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:17.710347 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36325014-5d55-43a5-ba3d-30e4a943c77d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:17.711705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36325014-5d55-43a5-ba3d-30e4a943c77d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:17.725327 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.sql.core [None req-36325014-5d55-43a5-ba3d-30e4a943c77d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 14:39:17.725327 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 19 14:39:17.725327 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [parameters: {'id': 'c934ad26568b468aacb083ef66737b65', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 19 14:39:17.725327 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87873) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 14:39:17.732552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 232/477] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:39:17 2022] POST /identity/v3/users => generated 249 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:39:17.748225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-36325014-5d55-43a5-ba3d-30e4a943c77d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 19 14:39:17.748225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:17.748225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 14:39:17.748225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 14:39:17.748225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 14:39:17.748225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 14:39:17.748225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 14:39:17.748225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 14:39:17.748225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 14:39:17.748225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application conn.query(q) Sep 19 14:39:17.748225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 14:39:17.748225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 14:39:17.748225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 14:39:17.748225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application result.read() Sep 19 14:39:17.748225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return ref.to_dict() Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 14:39:17.749199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application next(self.gen) Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application yield resource Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application next(self.gen) Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise self.value Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application session.commit() Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 14:39:17.750746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.session.flush() Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application compat.raise_( Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 14:39:17.751942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 14:39:17.752826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application conn.query(q) Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application result.read() Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 14:39:17.753749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [parameters: {'id': 'c934ad26568b468aacb083ef66737b65', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:17.754578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 19 14:39:17.755409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:17.758158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 246/478] 10.222.0.60 () {60 vars in 1131 bytes} [Mon Sep 19 14:39:17 2022] POST /identity/v3/roles => generated 144 bytes in 274 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:17.763867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03269e56-ff1d-4d73-8033-83513469b72a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:17.802897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:17.890072 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03269e56-ff1d-4d73-8033-83513469b72a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:17.928264 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225de902-5c29-4789-9a3b-b9c5eaf7237a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:17.928905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225de902-5c29-4789-9a3b-b9c5eaf7237a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:17.929187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225de902-5c29-4789-9a3b-b9c5eaf7237a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:17.929465 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225de902-5c29-4789-9a3b-b9c5eaf7237a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:17.940668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225de902-5c29-4789-9a3b-b9c5eaf7237a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-225de902-5c29-4789-9a3b-b9c5eaf7237a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:17.946065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:17.948235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:17.950853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:17.950853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:17.950853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:17.950853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:17.950853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:17.950853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:17.950853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:17.950853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:17.950853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:17.950853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:17.950853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 19 14:39:17.950853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:17.950853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 233/479] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:39:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:17.959865 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36325014-5d55-43a5-ba3d-30e4a943c77d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:18.118099 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36325014-5d55-43a5-ba3d-30e4a943c77d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:18.181859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018711e5-445a-4c3e-a43b-682570e9df1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:18.182890 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018711e5-445a-4c3e-a43b-682570e9df1e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:18.183381 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018711e5-445a-4c3e-a43b-682570e9df1e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:18.183895 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018711e5-445a-4c3e-a43b-682570e9df1e None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:18.184875 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018711e5-445a-4c3e-a43b-682570e9df1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:18.185397 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018711e5-445a-4c3e-a43b-682570e9df1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:18.187083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018711e5-445a-4c3e-a43b-682570e9df1e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:18.203192 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 247/480] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:19.135033 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a2031-bfd2-42cc-b2f7-fa0ab4eee6c6 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:19.135553 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a2031-bfd2-42cc-b2f7-fa0ab4eee6c6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:19.135553 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a2031-bfd2-42cc-b2f7-fa0ab4eee6c6 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:19.137207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 234/481] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:19.143115 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095714ce-439a-4203-928d-82cac9bf2a8c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:19.143406 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095714ce-439a-4203-928d-82cac9bf2a8c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:19.143683 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095714ce-439a-4203-928d-82cac9bf2a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:19.168944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-095714ce-439a-4203-928d-82cac9bf2a8c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:19.170280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-095714ce-439a-4203-928d-82cac9bf2a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:19.216716 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:19.275286 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 248/482] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:39:19.303985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87469d-d5dc-4ebe-992f-d9a1990ebd0d None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:19.304609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87469d-d5dc-4ebe-992f-d9a1990ebd0d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:19.305082 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc87469d-d5dc-4ebe-992f-d9a1990ebd0d None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:19.306873 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 235/483] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:19.313504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471621c-85a8-462f-afcc-dbc4e3ccfb64 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:19.313838 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471621c-85a8-462f-afcc-dbc4e3ccfb64 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:19.314060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471621c-85a8-462f-afcc-dbc4e3ccfb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:19.339292 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1471621c-85a8-462f-afcc-dbc4e3ccfb64 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:19.340426 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1471621c-85a8-462f-afcc-dbc4e3ccfb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:19.362498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc10b4-b57e-49fe-9bf1-64a34b31bddd None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:19.363038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc10b4-b57e-49fe-9bf1-64a34b31bddd None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:19.363461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc10b4-b57e-49fe-9bf1-64a34b31bddd None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:19.365177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 236/484] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:19.375054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eba5cc-9039-4d50-a2a3-5e17ac08ee84 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:19.375566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eba5cc-9039-4d50-a2a3-5e17ac08ee84 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:19.376053 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eba5cc-9039-4d50-a2a3-5e17ac08ee84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:19.407640 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-47eba5cc-9039-4d50-a2a3-5e17ac08ee84 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:19.409084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-47eba5cc-9039-4d50-a2a3-5e17ac08ee84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:19.441336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 249/485] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:39:19.496903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 237/486] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 19 14:39:19.604894 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471621c-85a8-462f-afcc-dbc4e3ccfb64 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:19.717710 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:19.725818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471621c-85a8-462f-afcc-dbc4e3ccfb64 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:19.746924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eba5cc-9039-4d50-a2a3-5e17ac08ee84 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:19.762391 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:19.781248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a63e60-f28b-4fe2-b269-3b3c4b157deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:19.783008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a63e60-f28b-4fe2-b269-3b3c4b157deb None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:19.784141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a63e60-f28b-4fe2-b269-3b3c4b157deb None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:19.784768 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a63e60-f28b-4fe2-b269-3b3c4b157deb None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:19.804044 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a63e60-f28b-4fe2-b269-3b3c4b157deb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-82a63e60-f28b-4fe2-b269-3b3c4b157deb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:19.814141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:19.815058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:19.815995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:19.815995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:19.815995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:19.815995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:19.815995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:19.815995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:19.815995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:19.815995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:19.815995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:19.815995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:19.815995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:19.815995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:19.818723 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 250/487] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:19.824021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a63e60-f28b-4fe2-b269-3b3c4b157deb None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:19.904599 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eba5cc-9039-4d50-a2a3-5e17ac08ee84 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:19.943502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a63e60-f28b-4fe2-b269-3b3c4b157deb None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:19.945217 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c288d-8cbf-45c3-921a-97b3e1ced195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:19.946166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c288d-8cbf-45c3-921a-97b3e1ced195 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:19.946677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c288d-8cbf-45c3-921a-97b3e1ced195 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:19.947149 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c288d-8cbf-45c3-921a-97b3e1ced195 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:19.958651 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c288d-8cbf-45c3-921a-97b3e1ced195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-b79c288d-8cbf-45c3-921a-97b3e1ced195 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:19.964137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:19.965047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:19.966392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:19.966392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:19.966392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:19.966392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:19.966392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:19.966392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:19.966392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:19.966392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:19.966392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:19.966392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:19.966392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:19.966392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:19.969233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 238/488] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:19.970272 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c288d-8cbf-45c3-921a-97b3e1ced195 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:19.985412 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea25c97-afb5-4363-afbf-aa15dc6c94af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:19.986245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea25c97-afb5-4363-afbf-aa15dc6c94af None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:19.986875 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea25c97-afb5-4363-afbf-aa15dc6c94af None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:19.987369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea25c97-afb5-4363-afbf-aa15dc6c94af None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:19.988583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea25c97-afb5-4363-afbf-aa15dc6c94af None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:19.988893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea25c97-afb5-4363-afbf-aa15dc6c94af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:19.991152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea25c97-afb5-4363-afbf-aa15dc6c94af None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:20.000473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 251/489] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:39:20.001704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea25c97-afb5-4363-afbf-aa15dc6c94af None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:20.022639 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:20.051691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:20.094236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c288d-8cbf-45c3-921a-97b3e1ced195 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:20.119762 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea25c97-afb5-4363-afbf-aa15dc6c94af None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:20.132653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14af3b7-2b63-41cc-ad47-1306d962b1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:20.133390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14af3b7-2b63-41cc-ad47-1306d962b1d8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:20.133812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14af3b7-2b63-41cc-ad47-1306d962b1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:20.134183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14af3b7-2b63-41cc-ad47-1306d962b1d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:20.145834 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14af3b7-2b63-41cc-ad47-1306d962b1d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-e14af3b7-2b63-41cc-ad47-1306d962b1d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:20.151014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:20.152054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:20.152885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:20.152885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:20.152885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:20.152885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:20.152885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:20.152885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:20.152885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:20.152885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:20.152885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:20.152885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:20.152885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:20.152885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:20.153892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 239/490] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:20.154422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14af3b7-2b63-41cc-ad47-1306d962b1d8 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:20.160542 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f39952-c131-46c0-8b28-330ed9892fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:20.161403 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f39952-c131-46c0-8b28-330ed9892fcc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:20.161858 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f39952-c131-46c0-8b28-330ed9892fcc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:20.162311 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f39952-c131-46c0-8b28-330ed9892fcc None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:20.163142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f39952-c131-46c0-8b28-330ed9892fcc None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:20.163635 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f39952-c131-46c0-8b28-330ed9892fcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:20.165232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f39952-c131-46c0-8b28-330ed9892fcc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:20.180034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 252/491] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:20.181188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f39952-c131-46c0-8b28-330ed9892fcc None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:20.264729 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14af3b7-2b63-41cc-ad47-1306d962b1d8 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:20.268615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:20.291021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f39952-c131-46c0-8b28-330ed9892fcc None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:20.298783 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:20.305101 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd651f-1258-43bc-a576-ad02c9d0ef45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:20.305609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd651f-1258-43bc-a576-ad02c9d0ef45 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:20.305956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd651f-1258-43bc-a576-ad02c9d0ef45 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:20.306297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd651f-1258-43bc-a576-ad02c9d0ef45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:20.317830 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd651f-1258-43bc-a576-ad02c9d0ef45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:20.319535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dd651f-1258-43bc-a576-ad02c9d0ef45 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:20.334369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7284339-4938-445a-958b-f1e4734c20f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:20.335250 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7284339-4938-445a-958b-f1e4734c20f0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:20.335736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7284339-4938-445a-958b-f1e4734c20f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:20.336253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7284339-4938-445a-958b-f1e4734c20f0 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:20.337090 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7284339-4938-445a-958b-f1e4734c20f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:20.337570 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7284339-4938-445a-958b-f1e4734c20f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:20.339101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7284339-4938-445a-958b-f1e4734c20f0 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-00dd651f-1258-43bc-a576-ad02c9d0ef45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:20.339969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:20.340941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:20.342304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:20.342304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:20.342304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:20.342304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 240/492] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:20.343445 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd651f-1258-43bc-a576-ad02c9d0ef45 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:20.352797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 253/493] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:20.354052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7284339-4938-445a-958b-f1e4734c20f0 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:20.454754 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dd651f-1258-43bc-a576-ad02c9d0ef45 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:20.463005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7284339-4938-445a-958b-f1e4734c20f0 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:20.492391 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aeb99e-2cbe-4314-b10b-be299ed42185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:20.493093 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aeb99e-2cbe-4314-b10b-be299ed42185 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:20.493441 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aeb99e-2cbe-4314-b10b-be299ed42185 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:20.493784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aeb99e-2cbe-4314-b10b-be299ed42185 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:20.500871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177f299-7e13-4ded-9fd4-e854b2ffe3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:20.501715 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177f299-7e13-4ded-9fd4-e854b2ffe3bf None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:20.502316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177f299-7e13-4ded-9fd4-e854b2ffe3bf None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:20.502775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177f299-7e13-4ded-9fd4-e854b2ffe3bf None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:20.503585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177f299-7e13-4ded-9fd4-e854b2ffe3bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:20.504180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177f299-7e13-4ded-9fd4-e854b2ffe3bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:20.505737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b177f299-7e13-4ded-9fd4-e854b2ffe3bf None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:20.506259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aeb99e-2cbe-4314-b10b-be299ed42185 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:20.507935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64aeb99e-2cbe-4314-b10b-be299ed42185 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:20.519404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 254/494] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:20.520655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177f299-7e13-4ded-9fd4-e854b2ffe3bf None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-64aeb99e-2cbe-4314-b10b-be299ed42185 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:20.523557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:20.524516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:20.530487 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:20.530487 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:20.530487 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:20.530487 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 241/495] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:20.530487 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aeb99e-2cbe-4314-b10b-be299ed42185 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:20.531635 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:20.567419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:20.644901 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b177f299-7e13-4ded-9fd4-e854b2ffe3bf None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:20.645101 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aeb99e-2cbe-4314-b10b-be299ed42185 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:20.682957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4858bf8-f2c0-4c5e-93ee-a27df639406d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:20.683189 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05527646-71cc-47ea-a974-3ca4c4f63d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:20.683780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05527646-71cc-47ea-a974-3ca4c4f63d93 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:20.684091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05527646-71cc-47ea-a974-3ca4c4f63d93 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:20.684414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05527646-71cc-47ea-a974-3ca4c4f63d93 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:20.685243 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4858bf8-f2c0-4c5e-93ee-a27df639406d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:20.685715 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4858bf8-f2c0-4c5e-93ee-a27df639406d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:20.686192 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4858bf8-f2c0-4c5e-93ee-a27df639406d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:20.693190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05527646-71cc-47ea-a974-3ca4c4f63d93 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:20.694834 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05527646-71cc-47ea-a974-3ca4c4f63d93 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:20.697808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4858bf8-f2c0-4c5e-93ee-a27df639406d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:20.699462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4858bf8-f2c0-4c5e-93ee-a27df639406d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-05527646-71cc-47ea-a974-3ca4c4f63d93 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:20.708344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:20.709293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:20.710218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:20.710218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:20.710218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:20.710218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:20.710218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:20.710218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:20.710218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:20.710218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:20.710218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:39:20.710218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:20.710218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 242/496] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Sep 19 14:39:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:20.710218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05527646-71cc-47ea-a974-3ca4c4f63d93 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-c4858bf8-f2c0-4c5e-93ee-a27df639406d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:20.720461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:20.721423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:20.722299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:20.722299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:20.722299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:20.724625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 255/497] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:20.725740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4858bf8-f2c0-4c5e-93ee-a27df639406d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:20.785741 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:20.808041 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:20.814214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05527646-71cc-47ea-a974-3ca4c4f63d93 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:20.838633 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4858bf8-f2c0-4c5e-93ee-a27df639406d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:20.848547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90950d39-5925-4813-83b9-4c0ca7e78292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:20.849158 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90950d39-5925-4813-83b9-4c0ca7e78292 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:20.849438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90950d39-5925-4813-83b9-4c0ca7e78292 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:20.849707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90950d39-5925-4813-83b9-4c0ca7e78292 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:20.850340 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90950d39-5925-4813-83b9-4c0ca7e78292 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:20.850641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90950d39-5925-4813-83b9-4c0ca7e78292 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:20.852045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90950d39-5925-4813-83b9-4c0ca7e78292 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:20.867877 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 243/498] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:20.867877 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90950d39-5925-4813-83b9-4c0ca7e78292 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:20.882768 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0862a-81da-4d95-b13e-b4183f050050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:20.883462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0862a-81da-4d95-b13e-b4183f050050 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:20.883737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0862a-81da-4d95-b13e-b4183f050050 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:20.884061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0862a-81da-4d95-b13e-b4183f050050 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:20.885020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0862a-81da-4d95-b13e-b4183f050050 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:20.885407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0862a-81da-4d95-b13e-b4183f050050 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:20.887387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a0862a-81da-4d95-b13e-b4183f050050 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:20.933740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 256/499] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Sep 19 14:39:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:20.935637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c58d98-fb25-485c-9d86-86a17a4a4995 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:20.936475 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c58d98-fb25-485c-9d86-86a17a4a4995 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:20.936475 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c58d98-fb25-485c-9d86-86a17a4a4995 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:20.941533 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 257/500] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:20.941533 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c58d98-fb25-485c-9d86-86a17a4a4995 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:20.980108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90950d39-5925-4813-83b9-4c0ca7e78292 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:21.016017 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7af698-dfe7-40fa-97dd-f15265931547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:21.016624 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7af698-dfe7-40fa-97dd-f15265931547 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:21.016903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7af698-dfe7-40fa-97dd-f15265931547 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:21.017178 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7af698-dfe7-40fa-97dd-f15265931547 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:21.017816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7af698-dfe7-40fa-97dd-f15265931547 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:21.018117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7af698-dfe7-40fa-97dd-f15265931547 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:21.019501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7af698-dfe7-40fa-97dd-f15265931547 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:21.030839 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 244/501] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:21.031634 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7af698-dfe7-40fa-97dd-f15265931547 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:21.048719 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c58d98-fb25-485c-9d86-86a17a4a4995 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:21.049914 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:21.063132 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:21.085125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2aa034-626a-44e4-ad53-2a716931450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:21.085751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2aa034-626a-44e4-ad53-2a716931450a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:21.086029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2aa034-626a-44e4-ad53-2a716931450a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:21.086294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2aa034-626a-44e4-ad53-2a716931450a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:21.094472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2aa034-626a-44e4-ad53-2a716931450a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:21.095952 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2aa034-626a-44e4-ad53-2a716931450a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-ca2aa034-626a-44e4-ad53-2a716931450a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:21.109247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:21.110380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:21.111382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:21.111382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:21.111382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:21.111382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:21.111382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:21.111382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:21.111382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:21.111382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:21.111382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 14:39:21.111382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:21.111382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 258/502] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:21.111992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21814156-f2a3-4b7a-9faa-e2f503e0f245 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:21.111992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21814156-f2a3-4b7a-9faa-e2f503e0f245 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:21.111992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21814156-f2a3-4b7a-9faa-e2f503e0f245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:21.129806 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-21814156-f2a3-4b7a-9faa-e2f503e0f245 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:21.130989 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-21814156-f2a3-4b7a-9faa-e2f503e0f245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:21.142072 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7af698-dfe7-40fa-97dd-f15265931547 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:21.176193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95309907-f77e-4a1b-9101-076d772c3e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:21.176824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95309907-f77e-4a1b-9101-076d772c3e67 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:21.177077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95309907-f77e-4a1b-9101-076d772c3e67 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:21.177348 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95309907-f77e-4a1b-9101-076d772c3e67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:21.197173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 259/503] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:21.198111 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16bd40-7c18-4e83-b7cb-72f3b772aea6 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:21.198401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16bd40-7c18-4e83-b7cb-72f3b772aea6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:21.198681 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16bd40-7c18-4e83-b7cb-72f3b772aea6 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:21.199917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 260/504] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:21.200613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16bd40-7c18-4e83-b7cb-72f3b772aea6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:21.258058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95309907-f77e-4a1b-9101-076d772c3e67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:21.259732 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95309907-f77e-4a1b-9101-076d772c3e67 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:21.276330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:21.308457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16bd40-7c18-4e83-b7cb-72f3b772aea6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:21.310370 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:21.358053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d425fb-4f30-4883-9096-c56145d216ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:21.359370 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d425fb-4f30-4883-9096-c56145d216ba None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:21.359984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d425fb-4f30-4883-9096-c56145d216ba None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:21.360568 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d425fb-4f30-4883-9096-c56145d216ba None admin] PATH_INFO: `/v3/users/glance` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:21.370722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 245/505] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:21.371843 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95309907-f77e-4a1b-9101-076d772c3e67 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:21.374104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d425fb-4f30-4883-9096-c56145d216ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:21.376571 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d425fb-4f30-4883-9096-c56145d216ba None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-f4d425fb-4f30-4883-9096-c56145d216ba None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:21.396448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:21.398210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:21.399127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:21.399127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:21.399127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:21.399127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:21.399127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:21.399127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:21.399127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:21.399127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:21.399127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 14:39:21.399127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:21.402505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 261/506] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:21.404331 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d822a5e-855c-47a8-a157-5e45c7e3e96b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:21.404653 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d822a5e-855c-47a8-a157-5e45c7e3e96b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:21.404927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d822a5e-855c-47a8-a157-5e45c7e3e96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:21.427215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9d822a5e-855c-47a8-a157-5e45c7e3e96b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:21.428952 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9d822a5e-855c-47a8-a157-5e45c7e3e96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:21.490887 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95309907-f77e-4a1b-9101-076d772c3e67 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:21.514151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 262/507] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:21.514151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d822a5e-855c-47a8-a157-5e45c7e3e96b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:21.514382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:21.530119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f67a27-ccf7-4d5b-aa1f-c7fc135c5f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:21.530884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f67a27-ccf7-4d5b-aa1f-c7fc135c5f8b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:21.531177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f67a27-ccf7-4d5b-aa1f-c7fc135c5f8b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:21.531878 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f67a27-ccf7-4d5b-aa1f-c7fc135c5f8b None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:21.532624 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f67a27-ccf7-4d5b-aa1f-c7fc135c5f8b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:21.532934 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f67a27-ccf7-4d5b-aa1f-c7fc135c5f8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:21.534388 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f67a27-ccf7-4d5b-aa1f-c7fc135c5f8b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:21.577653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 246/508] 10.222.0.60 () {56 vars in 1137 bytes} [Mon Sep 19 14:39:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:21.577653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f67a27-ccf7-4d5b-aa1f-c7fc135c5f8b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:21.627130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:21.636386 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d822a5e-855c-47a8-a157-5e45c7e3e96b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:21.673291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dd327-48a4-4f08-98db-c819c0702738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:21.673927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dd327-48a4-4f08-98db-c819c0702738 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:21.674216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dd327-48a4-4f08-98db-c819c0702738 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:21.674488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dd327-48a4-4f08-98db-c819c0702738 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:21.684844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dd327-48a4-4f08-98db-c819c0702738 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:21.686285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dd327-48a4-4f08-98db-c819c0702738 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:21.687871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f67a27-ccf7-4d5b-aa1f-c7fc135c5f8b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-f27dd327-48a4-4f08-98db-c819c0702738 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:21.701247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:21.702356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:21.703422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:21.703422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:21.703422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:21.706792 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 263/509] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:21.707561 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dd327-48a4-4f08-98db-c819c0702738 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:21.729017 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296a21e-6e02-45b7-a779-a798a924a826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:21.729837 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296a21e-6e02-45b7-a779-a798a924a826 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:21.730281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296a21e-6e02-45b7-a779-a798a924a826 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:21.730551 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296a21e-6e02-45b7-a779-a798a924a826 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:21.731274 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296a21e-6e02-45b7-a779-a798a924a826 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:21.731608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296a21e-6e02-45b7-a779-a798a924a826 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:21.733360 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296a21e-6e02-45b7-a779-a798a924a826 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:21.751697 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 247/510] 10.222.0.60 () {56 vars in 1137 bytes} [Mon Sep 19 14:39:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:21.752563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296a21e-6e02-45b7-a779-a798a924a826 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:21.785155 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:21.814994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dd327-48a4-4f08-98db-c819c0702738 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:21.850789 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc75d2-5260-40d4-86cb-73d605923d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:21.851542 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc75d2-5260-40d4-86cb-73d605923d71 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:21.851840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc75d2-5260-40d4-86cb-73d605923d71 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:21.852131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc75d2-5260-40d4-86cb-73d605923d71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:21.853176 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296a21e-6e02-45b7-a779-a798a924a826 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:21.862397 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc75d2-5260-40d4-86cb-73d605923d71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-a8fc75d2-5260-40d4-86cb-73d605923d71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:21.867642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:21.868636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:21.869429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:21.869429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:21.869429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:21.869429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:21.869429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:21.869429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:21.869429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:21.869429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:21.869429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:21.869429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:21.869429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:21.869429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:21.869429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 264/511] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:21.870609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc75d2-5260-40d4-86cb-73d605923d71 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:21.873754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:21.895324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03484754-a485-49cd-899d-22c5c4baa3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:21.895974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03484754-a485-49cd-899d-22c5c4baa3c6 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:21.896256 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03484754-a485-49cd-899d-22c5c4baa3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:21.896534 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03484754-a485-49cd-899d-22c5c4baa3c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:21.967545 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03484754-a485-49cd-899d-22c5c4baa3c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:21.969037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03484754-a485-49cd-899d-22c5c4baa3c6 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:21.980316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc75d2-5260-40d4-86cb-73d605923d71 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:21.999535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:22.015262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d83404-4802-44b7-a02b-5b6f78dc61ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:22.015939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d83404-4802-44b7-a02b-5b6f78dc61ea None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:22.016220 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d83404-4802-44b7-a02b-5b6f78dc61ea None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:22.016480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d83404-4802-44b7-a02b-5b6f78dc61ea None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:22.017113 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d83404-4802-44b7-a02b-5b6f78dc61ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:22.017420 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d83404-4802-44b7-a02b-5b6f78dc61ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:22.018807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d83404-4802-44b7-a02b-5b6f78dc61ea None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:22.031032 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 265/512] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:22.031859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d83404-4802-44b7-a02b-5b6f78dc61ea None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:22.073498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 248/513] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:22.076852 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03484754-a485-49cd-899d-22c5c4baa3c6 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:22.104924 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:22.146793 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d83404-4802-44b7-a02b-5b6f78dc61ea None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:22.174921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03484754-a485-49cd-899d-22c5c4baa3c6 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:22.180743 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc1172-4172-4bbc-bf20-27a64b43bd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:22.181358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc1172-4172-4bbc-bf20-27a64b43bd49 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:22.181638 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc1172-4172-4bbc-bf20-27a64b43bd49 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:22.181917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc1172-4172-4bbc-bf20-27a64b43bd49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:22.204686 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:22.210084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c6d84-104a-474a-afcb-5cc49380e856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:22.210723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c6d84-104a-474a-afcb-5cc49380e856 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:22.211001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c6d84-104a-474a-afcb-5cc49380e856 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:22.211519 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c6d84-104a-474a-afcb-5cc49380e856 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:22.222555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c6d84-104a-474a-afcb-5cc49380e856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:22.224115 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c6d84-104a-474a-afcb-5cc49380e856 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-d64c6d84-104a-474a-afcb-5cc49380e856 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:22.239612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:22.240773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:22.246899 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:22.246899 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:22.246899 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:22.246899 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 249/514] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:22.246899 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c6d84-104a-474a-afcb-5cc49380e856 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:22.260771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc1172-4172-4bbc-bf20-27a64b43bd49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:22.262248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc1172-4172-4bbc-bf20-27a64b43bd49 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:22.325907 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:22.341195 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c6d84-104a-474a-afcb-5cc49380e856 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:22.364802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 266/515] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:22.365610 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc1172-4172-4bbc-bf20-27a64b43bd49 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:22.379795 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef0e72-ea7d-4805-99b7-a1f8328c562a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:22.380440 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef0e72-ea7d-4805-99b7-a1f8328c562a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:22.380721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef0e72-ea7d-4805-99b7-a1f8328c562a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:22.380997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef0e72-ea7d-4805-99b7-a1f8328c562a None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:22.381745 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef0e72-ea7d-4805-99b7-a1f8328c562a None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:22.382051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef0e72-ea7d-4805-99b7-a1f8328c562a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:22.383452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef0e72-ea7d-4805-99b7-a1f8328c562a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:22.391434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 250/516] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:22.392258 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef0e72-ea7d-4805-99b7-a1f8328c562a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:22.442871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:22.469736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc1172-4172-4bbc-bf20-27a64b43bd49 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:22.498461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef0e72-ea7d-4805-99b7-a1f8328c562a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:22.503680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e43d1-b6ed-4dc2-a216-1e11f7cdb0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:22.504341 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e43d1-b6ed-4dc2-a216-1e11f7cdb0bb None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:22.504621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e43d1-b6ed-4dc2-a216-1e11f7cdb0bb None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:22.504892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e43d1-b6ed-4dc2-a216-1e11f7cdb0bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:22.511395 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e43d1-b6ed-4dc2-a216-1e11f7cdb0bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:22.512888 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e43d1-b6ed-4dc2-a216-1e11f7cdb0bb None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:22.524382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-d30e43d1-b6ed-4dc2-a216-1e11f7cdb0bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:22.524382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:22.524382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:22.524382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:22.524382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:22.524382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:22.524382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:22.524382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:22.524382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:22.524382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:22.524382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:22.524382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:22.524382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:22.524382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:22.524382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:22.525290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:22.526163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:22.526163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:22.526163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:22.526163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:22.526163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:22.526163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:22.526163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:22.526163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:22.526163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 267/517] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:22.526635 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e43d1-b6ed-4dc2-a216-1e11f7cdb0bb None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:22.535818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c2f8a-6400-4a1d-8c79-e6a04275c00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:22.536502 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c2f8a-6400-4a1d-8c79-e6a04275c00d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:22.536846 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c2f8a-6400-4a1d-8c79-e6a04275c00d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:22.537126 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c2f8a-6400-4a1d-8c79-e6a04275c00d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:22.548439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c2f8a-6400-4a1d-8c79-e6a04275c00d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:22.549911 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c2f8a-6400-4a1d-8c79-e6a04275c00d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:22.551274 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-512c2f8a-6400-4a1d-8c79-e6a04275c00d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:22.564496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:22.565437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:22.571106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:22.571106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:22.571106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:22.571106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 251/518] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:22.571106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c2f8a-6400-4a1d-8c79-e6a04275c00d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:22.628288 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e43d1-b6ed-4dc2-a216-1e11f7cdb0bb None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:22.665558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcd31a-0043-4761-8681-bb94670d13d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:22.666431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcd31a-0043-4761-8681-bb94670d13d4 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:22.666604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcd31a-0043-4761-8681-bb94670d13d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:22.666818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcd31a-0043-4761-8681-bb94670d13d4 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:22.667460 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcd31a-0043-4761-8681-bb94670d13d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:22.667857 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcd31a-0043-4761-8681-bb94670d13d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:22.668088 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:22.669280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcd31a-0043-4761-8681-bb94670d13d4 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:22.670605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c2f8a-6400-4a1d-8c79-e6a04275c00d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:22.682049 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 268/519] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:22.682881 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcd31a-0043-4761-8681-bb94670d13d4 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:22.710752 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508b8ca-47d8-452d-8f4d-567e4e38828f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:22.711377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508b8ca-47d8-452d-8f4d-567e4e38828f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:22.711653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508b8ca-47d8-452d-8f4d-567e4e38828f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:22.711958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508b8ca-47d8-452d-8f4d-567e4e38828f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:22.722170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508b8ca-47d8-452d-8f4d-567e4e38828f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:22.723619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508b8ca-47d8-452d-8f4d-567e4e38828f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-e508b8ca-47d8-452d-8f4d-567e4e38828f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:22.737533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:22.738483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:22.739338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:22.739338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:22.739338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:22.739338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 252/520] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:22.739338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508b8ca-47d8-452d-8f4d-567e4e38828f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:22.773043 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:22.789421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcd31a-0043-4761-8681-bb94670d13d4 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:22.824783 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb1803-e18a-4428-9536-053da150f4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:22.825577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb1803-e18a-4428-9536-053da150f4f9 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:22.825861 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb1803-e18a-4428-9536-053da150f4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:22.826136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb1803-e18a-4428-9536-053da150f4f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:22.836432 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb1803-e18a-4428-9536-053da150f4f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:22.838125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb1803-e18a-4428-9536-053da150f4f9 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:22.838793 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508b8ca-47d8-452d-8f4d-567e4e38828f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-15eb1803-e18a-4428-9536-053da150f4f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:22.853771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:22.854726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:22.855796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:22.855796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:22.855796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:22.855796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 269/521] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:22.857269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb1803-e18a-4428-9536-053da150f4f9 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:22.874497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb368c1-62f1-4efb-9b88-e8d6a78bd189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:22.875102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb368c1-62f1-4efb-9b88-e8d6a78bd189 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:22.875382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb368c1-62f1-4efb-9b88-e8d6a78bd189 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:22.875656 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb368c1-62f1-4efb-9b88-e8d6a78bd189 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:22.876389 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb368c1-62f1-4efb-9b88-e8d6a78bd189 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:22.877143 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb368c1-62f1-4efb-9b88-e8d6a78bd189 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:22.878149 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb368c1-62f1-4efb-9b88-e8d6a78bd189 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:22.895677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 253/522] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:22.896533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb368c1-62f1-4efb-9b88-e8d6a78bd189 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:22.908801 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:22.964658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb1803-e18a-4428-9536-053da150f4f9 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:22.997791 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb368c1-62f1-4efb-9b88-e8d6a78bd189 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:22.999159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92833e8-5ebc-4805-a041-122c6f5aaf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:22.999775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92833e8-5ebc-4805-a041-122c6f5aaf66 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:23.000096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92833e8-5ebc-4805-a041-122c6f5aaf66 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:23.000371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92833e8-5ebc-4805-a041-122c6f5aaf66 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:23.001007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92833e8-5ebc-4805-a041-122c6f5aaf66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:23.001311 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92833e8-5ebc-4805-a041-122c6f5aaf66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:23.002667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92833e8-5ebc-4805-a041-122c6f5aaf66 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:23.005432 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:23.015421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 270/523] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:23.016315 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92833e8-5ebc-4805-a041-122c6f5aaf66 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:23.033676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5bb2d-0ec8-49f1-9dde-9b882bf73139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:23.034281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5bb2d-0ec8-49f1-9dde-9b882bf73139 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:23.034564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5bb2d-0ec8-49f1-9dde-9b882bf73139 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:23.035140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5bb2d-0ec8-49f1-9dde-9b882bf73139 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:23.035791 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5bb2d-0ec8-49f1-9dde-9b882bf73139 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:23.037274 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5bb2d-0ec8-49f1-9dde-9b882bf73139 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:23.061091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 254/524] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:39:22 2022] POST /identity/v3/users => generated 247 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:39:23.061867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5bb2d-0ec8-49f1-9dde-9b882bf73139 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:23.127808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92833e8-5ebc-4805-a041-122c6f5aaf66 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:23.156169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:23.170603 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998635ac-0529-47db-9efa-f66bd792dc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:23.171210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998635ac-0529-47db-9efa-f66bd792dc7d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:23.171490 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998635ac-0529-47db-9efa-f66bd792dc7d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:23.171761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998635ac-0529-47db-9efa-f66bd792dc7d None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:23.172468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998635ac-0529-47db-9efa-f66bd792dc7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:23.172772 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998635ac-0529-47db-9efa-f66bd792dc7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:23.174416 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998635ac-0529-47db-9efa-f66bd792dc7d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:23.188127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 271/525] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:23.188788 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998635ac-0529-47db-9efa-f66bd792dc7d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:23.205695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5bb2d-0ec8-49f1-9dde-9b882bf73139 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:23.244770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853ca2a-720f-4ee6-96ec-b54c3543f6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:23.245393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853ca2a-720f-4ee6-96ec-b54c3543f6eb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:23.245674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853ca2a-720f-4ee6-96ec-b54c3543f6eb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:23.245965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853ca2a-720f-4ee6-96ec-b54c3543f6eb None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:23.246600 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853ca2a-720f-4ee6-96ec-b54c3543f6eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:23.246906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853ca2a-720f-4ee6-96ec-b54c3543f6eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:23.248944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853ca2a-720f-4ee6-96ec-b54c3543f6eb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:23.257204 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:23.263441 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 255/526] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:23.264339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853ca2a-720f-4ee6-96ec-b54c3543f6eb None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:23.315435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998635ac-0529-47db-9efa-f66bd792dc7d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:23.350967 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d861e-5c57-4bb0-9831-faa24f9432ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:23.351625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d861e-5c57-4bb0-9831-faa24f9432ca None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:23.351873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d861e-5c57-4bb0-9831-faa24f9432ca None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:23.352204 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d861e-5c57-4bb0-9831-faa24f9432ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:23.352877 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d861e-5c57-4bb0-9831-faa24f9432ca None admin] RBAC: Adding query filter params (role.id=15fbc4d519f64f4cacecc3b20fe826ae, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=59b5d2fd362b417c9b9be75f88a667ad) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:23.353193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d861e-5c57-4bb0-9831-faa24f9432ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:23.354627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d861e-5c57-4bb0-9831-faa24f9432ca None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:23.368280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 272/527] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:39:23 2022] GET /identity/v3/role_assignments?user.id=59b5d2fd362b417c9b9be75f88a667ad&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=15fbc4d519f64f4cacecc3b20fe826ae => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:23.369255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d861e-5c57-4bb0-9831-faa24f9432ca None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:23.379956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853ca2a-720f-4ee6-96ec-b54c3543f6eb None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:23.410975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:23.422589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b56f49-3629-4660-a730-3c5ec1b82844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:23.423650 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b56f49-3629-4660-a730-3c5ec1b82844 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:23.424097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b56f49-3629-4660-a730-3c5ec1b82844 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:23.424460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b56f49-3629-4660-a730-3c5ec1b82844 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:23.433019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b56f49-3629-4660-a730-3c5ec1b82844 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:23.435340 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b56f49-3629-4660-a730-3c5ec1b82844 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:23.455965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-e8b56f49-3629-4660-a730-3c5ec1b82844 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:23.455965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:23.455965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:23.455965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:23.455965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:23.455965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:23.455965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:23.455965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:23.455965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:23.455965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:23.455965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:23.455965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:23.455965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:23.455965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:23.455965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:23.457002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:23.457870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:23.457870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:23.457870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:23.457870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:23.457870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:23.457870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:23.457870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:23.457870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:23.460320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 256/528] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:23.461546 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b56f49-3629-4660-a730-3c5ec1b82844 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:23.516315 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d861e-5c57-4bb0-9831-faa24f9432ca None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:23.539055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:23.560561 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7192f-7dfb-4582-bd88-e4177596547d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:23.561209 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7192f-7dfb-4582-bd88-e4177596547d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:23.561487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7192f-7dfb-4582-bd88-e4177596547d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:23.561761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b7192f-7dfb-4582-bd88-e4177596547d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:23.570448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7192f-7dfb-4582-bd88-e4177596547d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:23.572282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b7192f-7dfb-4582-bd88-e4177596547d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:23.576173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b56f49-3629-4660-a730-3c5ec1b82844 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:23.589086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-f0b7192f-7dfb-4582-bd88-e4177596547d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:23.589086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:23.589086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:23.589086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:23.589086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:23.589086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:23.589086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:23.589086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:23.589086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:23.589086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:23.589086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:23.589086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:23.589086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:23.589086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:23.589086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:23.589986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:23.591461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:23.591461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:23.591461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:23.591461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:23.591461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:23.591461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:23.591461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:23.591461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:23.591461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 273/529] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:23.591978 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7192f-7dfb-4582-bd88-e4177596547d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:23.621128 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e422d7-d2ed-41d5-8427-5a49ca0535aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:23.622029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e422d7-d2ed-41d5-8427-5a49ca0535aa None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:23.622830 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e422d7-d2ed-41d5-8427-5a49ca0535aa None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:23.623106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e422d7-d2ed-41d5-8427-5a49ca0535aa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:23.633707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e422d7-d2ed-41d5-8427-5a49ca0535aa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:23.635214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e422d7-d2ed-41d5-8427-5a49ca0535aa None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-12e422d7-d2ed-41d5-8427-5a49ca0535aa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:23.650030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:23.651174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:23.652363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:23.652363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:23.652363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:23.652363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:23.652363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:23.652363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:23.652363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:23.652363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:23.652363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 14:39:23.652363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:23.655921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 257/530] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:23.656864 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e422d7-d2ed-41d5-8427-5a49ca0535aa None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:23.708902 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:23.724731 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b7192f-7dfb-4582-bd88-e4177596547d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:23.765926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25304e-a25a-45eb-8aa3-f27b4f46234b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:23.767259 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25304e-a25a-45eb-8aa3-f27b4f46234b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:23.768152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25304e-a25a-45eb-8aa3-f27b4f46234b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:23.768844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25304e-a25a-45eb-8aa3-f27b4f46234b None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:23.769680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25304e-a25a-45eb-8aa3-f27b4f46234b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:23.770178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25304e-a25a-45eb-8aa3-f27b4f46234b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:23.771897 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25304e-a25a-45eb-8aa3-f27b4f46234b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:23.776874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e422d7-d2ed-41d5-8427-5a49ca0535aa None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:23.793680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 274/531] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:23.795632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25304e-a25a-45eb-8aa3-f27b4f46234b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:23.819892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a80cb-a953-49c1-9273-c41de131a55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:23.820512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a80cb-a953-49c1-9273-c41de131a55e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:23.820796 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a80cb-a953-49c1-9273-c41de131a55e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:23.821071 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a80cb-a953-49c1-9273-c41de131a55e None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:23.821765 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a80cb-a953-49c1-9273-c41de131a55e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:23.822629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a80cb-a953-49c1-9273-c41de131a55e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:23.824435 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a80cb-a953-49c1-9273-c41de131a55e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:23.847156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 258/532] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:23.847156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a80cb-a953-49c1-9273-c41de131a55e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:23.853631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:23.930762 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25304e-a25a-45eb-8aa3-f27b4f46234b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:23.970455 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064110c-5fdb-4777-9542-3bac901c1494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:23.971468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064110c-5fdb-4777-9542-3bac901c1494 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:23.972264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064110c-5fdb-4777-9542-3bac901c1494 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:23.972540 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064110c-5fdb-4777-9542-3bac901c1494 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:23.973268 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064110c-5fdb-4777-9542-3bac901c1494 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:23.973572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064110c-5fdb-4777-9542-3bac901c1494 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:23.975000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064110c-5fdb-4777-9542-3bac901c1494 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:23.992538 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a80cb-a953-49c1-9273-c41de131a55e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:24.004129 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:24.011813 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 275/533] 10.222.0.60 () {56 vars in 1139 bytes} [Mon Sep 19 14:39:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:24.013300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064110c-5fdb-4777-9542-3bac901c1494 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:24.039260 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1288707-074f-4250-9c64-9ddbbc73fe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:24.040312 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1288707-074f-4250-9c64-9ddbbc73fe1f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:24.041642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1288707-074f-4250-9c64-9ddbbc73fe1f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:24.042143 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1288707-074f-4250-9c64-9ddbbc73fe1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:24.043393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1288707-074f-4250-9c64-9ddbbc73fe1f None admin] RBAC: Adding query filter params (role.id=15fbc4d519f64f4cacecc3b20fe826ae, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=006fe9ba3b0a40f29fa2be7620e548c4) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:24.044028 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1288707-074f-4250-9c64-9ddbbc73fe1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:24.046687 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1288707-074f-4250-9c64-9ddbbc73fe1f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:24.055098 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 259/534] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:39:23 2022] GET /identity/v3/role_assignments?user.id=006fe9ba3b0a40f29fa2be7620e548c4&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=15fbc4d519f64f4cacecc3b20fe826ae => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:24.056365 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1288707-074f-4250-9c64-9ddbbc73fe1f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:24.128713 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:24.132881 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064110c-5fdb-4777-9542-3bac901c1494 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:24.172104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8e180-5ae7-4448-abeb-ba31d973426a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:24.172724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8e180-5ae7-4448-abeb-ba31d973426a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:24.173006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8e180-5ae7-4448-abeb-ba31d973426a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:24.173530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8e180-5ae7-4448-abeb-ba31d973426a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:24.174350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8e180-5ae7-4448-abeb-ba31d973426a None admin] RBAC: Adding query filter params (role.id=15fbc4d519f64f4cacecc3b20fe826ae, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=caab7aaa72634b358d717768fbcb48d8) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:24.174659 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8e180-5ae7-4448-abeb-ba31d973426a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:24.176248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f8e180-5ae7-4448-abeb-ba31d973426a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:24.186336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1288707-074f-4250-9c64-9ddbbc73fe1f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:24.188011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 276/535] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:39:24 2022] GET /identity/v3/role_assignments?user.id=caab7aaa72634b358d717768fbcb48d8&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=15fbc4d519f64f4cacecc3b20fe826ae => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:24.229552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3f13a-6f79-41f6-9b1f-80447d3f5f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:24.230917 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3f13a-6f79-41f6-9b1f-80447d3f5f69 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:24.231416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3f13a-6f79-41f6-9b1f-80447d3f5f69 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:24.231960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb3f13a-6f79-41f6-9b1f-80447d3f5f69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:24.280466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:24.317895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3f13a-6f79-41f6-9b1f-80447d3f5f69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:24.319936 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb3f13a-6f79-41f6-9b1f-80447d3f5f69 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:24.439006 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 260/536] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:24.444851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8e180-5ae7-4448-abeb-ba31d973426a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:24.562781 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8e180-5ae7-4448-abeb-ba31d973426a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:24.598193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4e5c4-1712-4f2f-8c33-7b05619d9fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:24.598830 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4e5c4-1712-4f2f-8c33-7b05619d9fa8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:24.599154 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4e5c4-1712-4f2f-8c33-7b05619d9fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:24.599379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4e5c4-1712-4f2f-8c33-7b05619d9fa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:24.611450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4e5c4-1712-4f2f-8c33-7b05619d9fa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:24.612988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4e5c4-1712-4f2f-8c33-7b05619d9fa8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-5ae4e5c4-1712-4f2f-8c33-7b05619d9fa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:24.628217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:24.629189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:24.630127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:24.630127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:24.630127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:24.630127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 277/537] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:24.634925 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3f13a-6f79-41f6-9b1f-80447d3f5f69 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:24.748881 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:24.790539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb3f13a-6f79-41f6-9b1f-80447d3f5f69 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:24.843762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4aba98-f7e6-4b0c-bc3f-ed3e56c253e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:24.844130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4aba98-f7e6-4b0c-bc3f-ed3e56c253e0 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:24.844314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4aba98-f7e6-4b0c-bc3f-ed3e56c253e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:24.844591 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4aba98-f7e6-4b0c-bc3f-ed3e56c253e0 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:24.845241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4aba98-f7e6-4b0c-bc3f-ed3e56c253e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:24.845544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4aba98-f7e6-4b0c-bc3f-ed3e56c253e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:24.846931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4aba98-f7e6-4b0c-bc3f-ed3e56c253e0 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:24.863343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 261/538] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:24.868811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4e5c4-1712-4f2f-8c33-7b05619d9fa8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:24.895636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:25.010958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4e5c4-1712-4f2f-8c33-7b05619d9fa8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:25.065555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1f58a-b9f2-4749-84a3-8f6262afd643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:25.066343 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1f58a-b9f2-4749-84a3-8f6262afd643 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:25.066632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1f58a-b9f2-4749-84a3-8f6262afd643 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:25.066909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1f58a-b9f2-4749-84a3-8f6262afd643 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:25.074773 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1f58a-b9f2-4749-84a3-8f6262afd643 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:25.076738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1f58a-b9f2-4749-84a3-8f6262afd643 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:25.091876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-c0c1f58a-b9f2-4749-84a3-8f6262afd643 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:25.091876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:25.091876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:25.091876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:25.091876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:25.091876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:25.091876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:25.091876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:25.091876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:25.091876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:25.091876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:25.091876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:25.091876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:25.091876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:25.091876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:25.092761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:25.093601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:25.093601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:25.093601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:25.093601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:25.093601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:25.093601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:25.093601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:25.093601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:25.097085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 278/539] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:25.102861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4aba98-f7e6-4b0c-bc3f-ed3e56c253e0 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:25.266542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4aba98-f7e6-4b0c-bc3f-ed3e56c253e0 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:25.311851 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:25.323202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1696faa-5ca5-439e-9503-410c49768623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:25.323833 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1696faa-5ca5-439e-9503-410c49768623 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:25.324136 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1696faa-5ca5-439e-9503-410c49768623 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:25.324412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1696faa-5ca5-439e-9503-410c49768623 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:25.325105 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1696faa-5ca5-439e-9503-410c49768623 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:25.325932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1696faa-5ca5-439e-9503-410c49768623 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:25.327399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1696faa-5ca5-439e-9503-410c49768623 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:25.343547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 262/540] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:25.349530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1f58a-b9f2-4749-84a3-8f6262afd643 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:25.474240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:25.504945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1f58a-b9f2-4749-84a3-8f6262afd643 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:25.558456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e9266-8420-4a0a-a705-1b61788b9334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:25.559124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e9266-8420-4a0a-a705-1b61788b9334 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:25.559412 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e9266-8420-4a0a-a705-1b61788b9334 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:25.559714 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e9266-8420-4a0a-a705-1b61788b9334 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:25.560467 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e9266-8420-4a0a-a705-1b61788b9334 None admin] RBAC: Adding query filter params (role.id=15fbc4d519f64f4cacecc3b20fe826ae, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=2fbef7b24f134f5eb1193aed1b7ee69c) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:25.560781 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e9266-8420-4a0a-a705-1b61788b9334 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:25.562200 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e9266-8420-4a0a-a705-1b61788b9334 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:25.566804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e088a-3eef-48b1-a5ae-c90f8fbc55b3 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:25.567091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e088a-3eef-48b1-a5ae-c90f8fbc55b3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:25.567378 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e088a-3eef-48b1-a5ae-c90f8fbc55b3 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:25.569002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 263/541] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:39:25.569923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 279/542] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:39:25 2022] GET /identity/v3/role_assignments?user.id=2fbef7b24f134f5eb1193aed1b7ee69c&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=15fbc4d519f64f4cacecc3b20fe826ae => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:25.574468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d3dc8-5a65-44cb-bdcd-d06eb647f32d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:25.574751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d3dc8-5a65-44cb-bdcd-d06eb647f32d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:25.575010 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d3dc8-5a65-44cb-bdcd-d06eb647f32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:25.595468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e47d3dc8-5a65-44cb-bdcd-d06eb647f32d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:25.596771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e47d3dc8-5a65-44cb-bdcd-d06eb647f32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:25.677732 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 264/543] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:25.968176 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e44986-d588-4901-bbdf-4327b137e0d9 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:25.968534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e44986-d588-4901-bbdf-4327b137e0d9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:25.968709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e44986-d588-4901-bbdf-4327b137e0d9 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:25.970283 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 280/544] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:25.975809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42af98-9b57-46e4-958c-99296395e2ef None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:25.976115 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42af98-9b57-46e4-958c-99296395e2ef None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:25.976393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42af98-9b57-46e4-958c-99296395e2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:25.994986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3f42af98-9b57-46e4-958c-99296395e2ef None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:25.996217 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3f42af98-9b57-46e4-958c-99296395e2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:26.076003 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 265/545] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:26.105841 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e44986-d588-4901-bbdf-4327b137e0d9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:26.239099 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e44986-d588-4901-bbdf-4327b137e0d9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:26.277609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9528e3-fbc6-4ed1-9d87-8965d5e3c4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:26.278249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9528e3-fbc6-4ed1-9d87-8965d5e3c4f7 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:26.278581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9528e3-fbc6-4ed1-9d87-8965d5e3c4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:26.278854 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9528e3-fbc6-4ed1-9d87-8965d5e3c4f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:26.289488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:26.294542 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9528e3-fbc6-4ed1-9d87-8965d5e3c4f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-af9528e3-fbc6-4ed1-9d87-8965d5e3c4f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:26.299753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:26.300655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:26.301462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:26.301462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:26.301462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:26.301462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:26.301462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:26.301462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:26.301462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:26.301462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:26.301462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:26.301462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:26.301462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:26.301462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:26.304292 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 281/546] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:26.309834 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42af98-9b57-46e4-958c-99296395e2ef None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:26.395436 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:26.460794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42af98-9b57-46e4-958c-99296395e2ef None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:26.496508 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9528e3-fbc6-4ed1-9d87-8965d5e3c4f7 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:26.509809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d4788-3ffc-4f0b-8531-3fc58cc332da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:26.510748 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d4788-3ffc-4f0b-8531-3fc58cc332da None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:26.511167 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d4788-3ffc-4f0b-8531-3fc58cc332da None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:26.511588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d4788-3ffc-4f0b-8531-3fc58cc332da None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:26.512290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d4788-3ffc-4f0b-8531-3fc58cc332da None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:26.512592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d4788-3ffc-4f0b-8531-3fc58cc332da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:26.514147 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d4788-3ffc-4f0b-8531-3fc58cc332da None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:26.527489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 266/547] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:26.534540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d4788-3ffc-4f0b-8531-3fc58cc332da None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:26.688619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9528e3-fbc6-4ed1-9d87-8965d5e3c4f7 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:26.721711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d4788-3ffc-4f0b-8531-3fc58cc332da None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:26.744043 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:26.753724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72372099-81d0-4927-bb9d-ee4a585f2d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:26.754316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72372099-81d0-4927-bb9d-ee4a585f2d6e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:26.754625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72372099-81d0-4927-bb9d-ee4a585f2d6e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:26.754975 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72372099-81d0-4927-bb9d-ee4a585f2d6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:26.776643 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72372099-81d0-4927-bb9d-ee4a585f2d6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-72372099-81d0-4927-bb9d-ee4a585f2d6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:26.783996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:26.784892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:26.785699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:26.785699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:26.785699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:26.785699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:26.785699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:26.785699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:26.785699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:26.785699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:26.785699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:26.785699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:26.785699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:26.785699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:26.790230 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 282/548] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:39:26.791439 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72372099-81d0-4927-bb9d-ee4a585f2d6e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:26.792884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85adff-cd1a-4687-98a2-2f5f9afc2262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:26.793531 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85adff-cd1a-4687-98a2-2f5f9afc2262 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:26.793810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85adff-cd1a-4687-98a2-2f5f9afc2262 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:26.794110 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab85adff-cd1a-4687-98a2-2f5f9afc2262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:26.814207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85adff-cd1a-4687-98a2-2f5f9afc2262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:26.815749 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab85adff-cd1a-4687-98a2-2f5f9afc2262 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-ab85adff-cd1a-4687-98a2-2f5f9afc2262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:26.847253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:26.849166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:26.850469 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:26.850469 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:26.850469 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:26.850469 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 267/549] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:39:26.858781 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85adff-cd1a-4687-98a2-2f5f9afc2262 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:26.873996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:26.947865 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72372099-81d0-4927-bb9d-ee4a585f2d6e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:26.995700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3936587-022e-41d5-b51d-d0b1d64509b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:26.996350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3936587-022e-41d5-b51d-d0b1d64509b4 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:26.996631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3936587-022e-41d5-b51d-d0b1d64509b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:26.996900 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3936587-022e-41d5-b51d-d0b1d64509b4 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:26.997841 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3936587-022e-41d5-b51d-d0b1d64509b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:26.998016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3936587-022e-41d5-b51d-d0b1d64509b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:26.999442 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3936587-022e-41d5-b51d-d0b1d64509b4 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:27.008125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 283/550] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:27.009469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee57df8-a521-4eaa-8e55-fa162c26c92b None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:27.009469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee57df8-a521-4eaa-8e55-fa162c26c92b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:27.009677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee57df8-a521-4eaa-8e55-fa162c26c92b None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:27.012223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 284/551] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:27.012223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097c1fe-be61-485f-8505-7b6a060806e3 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:27.012414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097c1fe-be61-485f-8505-7b6a060806e3 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:27.012536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097c1fe-be61-485f-8505-7b6a060806e3 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:27.014203 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 285/552] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:27.015687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097c1fe-be61-485f-8505-7b6a060806e3 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:27.037461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab85adff-cd1a-4687-98a2-2f5f9afc2262 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:27.074344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7300dd-f16b-4c6f-afae-116cfa8e5eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:27.074992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7300dd-f16b-4c6f-afae-116cfa8e5eae None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:27.075747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7300dd-f16b-4c6f-afae-116cfa8e5eae None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:27.076050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7300dd-f16b-4c6f-afae-116cfa8e5eae None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:27.076688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7300dd-f16b-4c6f-afae-116cfa8e5eae None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:27.076989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7300dd-f16b-4c6f-afae-116cfa8e5eae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:27.078359 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7300dd-f16b-4c6f-afae-116cfa8e5eae None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:27.090786 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 268/553] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:27.092668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae40b8-98db-4cf6-aa1f-1d457cc46b9c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:27.092958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae40b8-98db-4cf6-aa1f-1d457cc46b9c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:27.093255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae40b8-98db-4cf6-aa1f-1d457cc46b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:27.112055 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-4eae40b8-98db-4cf6-aa1f-1d457cc46b9c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:27.113136 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4eae40b8-98db-4cf6-aa1f-1d457cc46b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:27.133440 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:27.145442 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097c1fe-be61-485f-8505-7b6a060806e3 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:27.166435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:27.182316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbe4e5-4118-4f07-a7e9-98ee084c4f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:27.183248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbe4e5-4118-4f07-a7e9-98ee084c4f54 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:27.183528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbe4e5-4118-4f07-a7e9-98ee084c4f54 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:27.183801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbe4e5-4118-4f07-a7e9-98ee084c4f54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:27.187745 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 269/554] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:27.188884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4500cb-d89e-4dd5-bcc6-07437104c5f9 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:27.189166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4500cb-d89e-4dd5-bcc6-07437104c5f9 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:27.189433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4500cb-d89e-4dd5-bcc6-07437104c5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:27.202956 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbe4e5-4118-4f07-a7e9-98ee084c4f54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:27.205099 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbe4e5-4118-4f07-a7e9-98ee084c4f54 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:27.207612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1f4500cb-d89e-4dd5-bcc6-07437104c5f9 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:27.208824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1f4500cb-d89e-4dd5-bcc6-07437104c5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-91bbe4e5-4118-4f07-a7e9-98ee084c4f54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:27.223991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:27.224980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:27.225864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:27.225864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:27.225864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:27.232479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 286/555] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:39:27.234386 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbe4e5-4118-4f07-a7e9-98ee084c4f54 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:27.291738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 270/556] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:27.292932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4500cb-d89e-4dd5-bcc6-07437104c5f9 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:27.370931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbe4e5-4118-4f07-a7e9-98ee084c4f54 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:27.384702 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:27.404133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4500cb-d89e-4dd5-bcc6-07437104c5f9 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:27.432831 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae27e38-d002-4c18-aa6f-40d5b7064039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:27.433840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae27e38-d002-4c18-aa6f-40d5b7064039 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:27.434069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae27e38-d002-4c18-aa6f-40d5b7064039 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:27.434515 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae27e38-d002-4c18-aa6f-40d5b7064039 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:27.442766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75f776-1c9f-4bdd-98e8-fbdfedba93ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:27.443718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75f776-1c9f-4bdd-98e8-fbdfedba93ba None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:27.444238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75f776-1c9f-4bdd-98e8-fbdfedba93ba None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:27.444707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75f776-1c9f-4bdd-98e8-fbdfedba93ba None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:27.445557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75f776-1c9f-4bdd-98e8-fbdfedba93ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:27.445663 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae27e38-d002-4c18-aa6f-40d5b7064039 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:27.446161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75f776-1c9f-4bdd-98e8-fbdfedba93ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:27.447136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae27e38-d002-4c18-aa6f-40d5b7064039 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:27.448102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75f776-1c9f-4bdd-98e8-fbdfedba93ba None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:27.464923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 271/557] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:27.466787 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75f776-1c9f-4bdd-98e8-fbdfedba93ba None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-0ae27e38-d002-4c18-aa6f-40d5b7064039 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:27.467048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:27.467951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:27.469229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:27.469229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:27.469229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:27.469229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:27.469229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:27.469229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:27.469229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:27.469229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:27.469229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 14:39:27.469229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:27.469229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 287/558] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:39:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:27.475236 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae27e38-d002-4c18-aa6f-40d5b7064039 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:27.508963 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:27.581756 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75f776-1c9f-4bdd-98e8-fbdfedba93ba None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:27.583216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae27e38-d002-4c18-aa6f-40d5b7064039 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:27.619552 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a2cfa-5517-4d7a-b07b-bc0b670388b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:27.620273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a2cfa-5517-4d7a-b07b-bc0b670388b3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:27.620618 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a2cfa-5517-4d7a-b07b-bc0b670388b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:27.620889 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a2cfa-5517-4d7a-b07b-bc0b670388b3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:27.622780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908ecdd-c299-4dd2-9622-065af1edc377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:27.623796 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908ecdd-c299-4dd2-9622-065af1edc377 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:27.624513 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908ecdd-c299-4dd2-9622-065af1edc377 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:27.624988 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908ecdd-c299-4dd2-9622-065af1edc377 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:27.629917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a2cfa-5517-4d7a-b07b-bc0b670388b3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:27.631420 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a2cfa-5517-4d7a-b07b-bc0b670388b3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:27.637803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908ecdd-c299-4dd2-9622-065af1edc377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-6908ecdd-c299-4dd2-9622-065af1edc377 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:27.643363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:27.644338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:27.650270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:27.650270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:27.650270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:27.650270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:27.650270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:27.650270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:27.650270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:27.650270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:27.650270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:27.650270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:27.650270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:27.650270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-071a2cfa-5517-4d7a-b07b-bc0b670388b3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:27.650922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:27.651782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:27.652839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:27.652839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:27.652839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:27.652839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:27.652839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:27.652839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:27.652839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:27.652839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:27.652839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:39:27.652839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:27.652839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 288/559] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Sep 19 14:39:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:27.652839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a2cfa-5517-4d7a-b07b-bc0b670388b3 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:27.654907 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 272/560] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:27.656807 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908ecdd-c299-4dd2-9622-065af1edc377 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:27.661556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:27.749788 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:27.759562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a2cfa-5517-4d7a-b07b-bc0b670388b3 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:27.762185 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908ecdd-c299-4dd2-9622-065af1edc377 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:27.796703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13027d54-aa36-4c38-a77e-2aa0b9bd2fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:27.797301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13027d54-aa36-4c38-a77e-2aa0b9bd2fdf None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:27.797587 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13027d54-aa36-4c38-a77e-2aa0b9bd2fdf None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:27.797858 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13027d54-aa36-4c38-a77e-2aa0b9bd2fdf None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:27.798734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad3eaa-27ff-45d2-83d1-bd474205597e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:27.798854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13027d54-aa36-4c38-a77e-2aa0b9bd2fdf None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:27.799050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13027d54-aa36-4c38-a77e-2aa0b9bd2fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:27.799741 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad3eaa-27ff-45d2-83d1-bd474205597e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:27.800179 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad3eaa-27ff-45d2-83d1-bd474205597e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:27.800460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13027d54-aa36-4c38-a77e-2aa0b9bd2fdf None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:27.800757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad3eaa-27ff-45d2-83d1-bd474205597e None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:27.801614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad3eaa-27ff-45d2-83d1-bd474205597e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:27.802206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad3eaa-27ff-45d2-83d1-bd474205597e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:27.804157 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad3eaa-27ff-45d2-83d1-bd474205597e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:27.808788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 273/561] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:27.809642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13027d54-aa36-4c38-a77e-2aa0b9bd2fdf None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:27.824895 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 289/562] 10.222.0.60 () {56 vars in 1135 bytes} [Mon Sep 19 14:39:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:27.826485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad3eaa-27ff-45d2-83d1-bd474205597e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:27.888776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:27.927769 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13027d54-aa36-4c38-a77e-2aa0b9bd2fdf None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:27.940569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad3eaa-27ff-45d2-83d1-bd474205597e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:27.962865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc61218-0665-464d-a0d7-ffebef9e47e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:27.963481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc61218-0665-464d-a0d7-ffebef9e47e6 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:27.963769 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc61218-0665-464d-a0d7-ffebef9e47e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:27.964073 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc61218-0665-464d-a0d7-ffebef9e47e6 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:27.964802 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc61218-0665-464d-a0d7-ffebef9e47e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:27.965107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc61218-0665-464d-a0d7-ffebef9e47e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:27.967190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc61218-0665-464d-a0d7-ffebef9e47e6 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:27.978572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071cf69c-c9b3-4d6e-a45b-0faec5144579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:27.979425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071cf69c-c9b3-4d6e-a45b-0faec5144579 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:27.979880 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071cf69c-c9b3-4d6e-a45b-0faec5144579 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:27.980307 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071cf69c-c9b3-4d6e-a45b-0faec5144579 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:27.986602 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 274/563] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Sep 19 14:39:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:27.987341 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc61218-0665-464d-a0d7-ffebef9e47e6 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:27.997903 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071cf69c-c9b3-4d6e-a45b-0faec5144579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-071cf69c-c9b3-4d6e-a45b-0faec5144579 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:28.002709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:28.003786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:28.004668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:28.004668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:28.004668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:28.004668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:28.004668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:28.004668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:28.004668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:28.004668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:28.004668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:28.004668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:28.004668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:28.004668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:28.004668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 290/564] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:28.004668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071cf69c-c9b3-4d6e-a45b-0faec5144579 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:28.017115 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:28.106836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc61218-0665-464d-a0d7-ffebef9e47e6 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:28.119352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071cf69c-c9b3-4d6e-a45b-0faec5144579 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:28.132870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:28.145204 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa3812-306e-46a1-9c14-6c430915ae03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:28.145893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa3812-306e-46a1-9c14-6c430915ae03 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:28.146165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa3812-306e-46a1-9c14-6c430915ae03 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:28.146430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa3812-306e-46a1-9c14-6c430915ae03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:28.156960 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5a9fd-2007-42fd-b8d7-e2394430bcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:28.157577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5a9fd-2007-42fd-b8d7-e2394430bcdf None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:28.157872 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5a9fd-2007-42fd-b8d7-e2394430bcdf None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:28.158134 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5a9fd-2007-42fd-b8d7-e2394430bcdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:28.158251 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa3812-306e-46a1-9c14-6c430915ae03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:28.159680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa3812-306e-46a1-9c14-6c430915ae03 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-fbaa3812-306e-46a1-9c14-6c430915ae03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:28.182065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:28.183316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:28.184475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:28.184475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:28.184475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:28.184475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 275/565] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:28.184475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa3812-306e-46a1-9c14-6c430915ae03 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:28.238031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f5a9fd-2007-42fd-b8d7-e2394430bcdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:28.239523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f5a9fd-2007-42fd-b8d7-e2394430bcdf None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:28.250399 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:28.282933 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa3812-306e-46a1-9c14-6c430915ae03 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:28.315777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7eeb25-2a4f-4ba8-96a0-c95e32b77c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:28.316429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7eeb25-2a4f-4ba8-96a0-c95e32b77c2b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:28.316706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7eeb25-2a4f-4ba8-96a0-c95e32b77c2b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:28.316973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7eeb25-2a4f-4ba8-96a0-c95e32b77c2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:28.328091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7eeb25-2a4f-4ba8-96a0-c95e32b77c2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:28.329532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7eeb25-2a4f-4ba8-96a0-c95e32b77c2b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:28.337488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 291/566] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:28.338337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5a9fd-2007-42fd-b8d7-e2394430bcdf None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-ab7eeb25-2a4f-4ba8-96a0-c95e32b77c2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:28.347326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:28.348361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:28.349269 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:28.349269 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:28.349269 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:28.349269 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 276/567] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:28.349269 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7eeb25-2a4f-4ba8-96a0-c95e32b77c2b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:28.380555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:28.446540 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5a9fd-2007-42fd-b8d7-e2394430bcdf None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:28.454716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7eeb25-2a4f-4ba8-96a0-c95e32b77c2b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:28.457771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:28.482437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806e5ca-5f65-4744-b2a9-272465a7ecde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:28.483049 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806e5ca-5f65-4744-b2a9-272465a7ecde None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:28.483333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806e5ca-5f65-4744-b2a9-272465a7ecde None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:28.483611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806e5ca-5f65-4744-b2a9-272465a7ecde None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:28.484375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806e5ca-5f65-4744-b2a9-272465a7ecde None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:28.484602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806e5ca-5f65-4744-b2a9-272465a7ecde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:28.485964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806e5ca-5f65-4744-b2a9-272465a7ecde None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:28.487999 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cc024-8305-43e8-90b8-f03b8af93d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:28.488599 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cc024-8305-43e8-90b8-f03b8af93d06 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:28.489151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cc024-8305-43e8-90b8-f03b8af93d06 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:28.489427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cc024-8305-43e8-90b8-f03b8af93d06 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:28.490055 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cc024-8305-43e8-90b8-f03b8af93d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:28.490359 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cc024-8305-43e8-90b8-f03b8af93d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:28.491708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cc024-8305-43e8-90b8-f03b8af93d06 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:28.494222 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 292/568] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:28.495204 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f21ce9-c824-4e85-8561-e5149d33c056 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:28.495490 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f21ce9-c824-4e85-8561-e5149d33c056 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:28.495767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f21ce9-c824-4e85-8561-e5149d33c056 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:28.499877 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 293/569] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:28.499877 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f21ce9-c824-4e85-8561-e5149d33c056 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:28.508214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 277/570] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:28.508214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cc024-8305-43e8-90b8-f03b8af93d06 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:28.592168 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:28.603735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cc024-8305-43e8-90b8-f03b8af93d06 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:28.604018 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f21ce9-c824-4e85-8561-e5149d33c056 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:28.638578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1093e78-55f8-4d2f-aafa-b232b973c529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:28.639012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac544b-31f7-4255-9a35-129e11ae9eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:28.639225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1093e78-55f8-4d2f-aafa-b232b973c529 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:28.639505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1093e78-55f8-4d2f-aafa-b232b973c529 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:28.639604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac544b-31f7-4255-9a35-129e11ae9eb2 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:28.639773 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1093e78-55f8-4d2f-aafa-b232b973c529 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:28.639911 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac544b-31f7-4255-9a35-129e11ae9eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:28.640179 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac544b-31f7-4255-9a35-129e11ae9eb2 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:28.640815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac544b-31f7-4255-9a35-129e11ae9eb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:28.641117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac544b-31f7-4255-9a35-129e11ae9eb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:28.642498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac544b-31f7-4255-9a35-129e11ae9eb2 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:28.652536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1093e78-55f8-4d2f-aafa-b232b973c529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:28.653929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 278/571] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:28.654628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1093e78-55f8-4d2f-aafa-b232b973c529 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:28.654913 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac544b-31f7-4255-9a35-129e11ae9eb2 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-b1093e78-55f8-4d2f-aafa-b232b973c529 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:28.675577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:28.676921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:28.678492 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:28.678492 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:28.678492 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:28.678492 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 294/572] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:28.678492 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711ab6a-a3bf-43ac-8013-f424b67bdb84 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:28.678492 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711ab6a-a3bf-43ac-8013-f424b67bdb84 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:28.679117 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711ab6a-a3bf-43ac-8013-f424b67bdb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:28.702671 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2711ab6a-a3bf-43ac-8013-f424b67bdb84 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:28.703790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2711ab6a-a3bf-43ac-8013-f424b67bdb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:28.725794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:28.766319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac544b-31f7-4255-9a35-129e11ae9eb2 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:28.777125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 295/573] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:28.777758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711ab6a-a3bf-43ac-8013-f424b67bdb84 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:28.817259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050f037-4798-4900-bc9f-8438ab00306a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:28.817935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050f037-4798-4900-bc9f-8438ab00306a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:28.818209 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050f037-4798-4900-bc9f-8438ab00306a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:28.818482 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050f037-4798-4900-bc9f-8438ab00306a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:28.832327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050f037-4798-4900-bc9f-8438ab00306a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:28.833815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050f037-4798-4900-bc9f-8438ab00306a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:28.842967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-9050f037-4798-4900-bc9f-8438ab00306a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:28.850290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:28.851254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:28.852193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:28.852193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:28.852193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:28.858625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 279/574] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:28.858625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050f037-4798-4900-bc9f-8438ab00306a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:28.894690 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711ab6a-a3bf-43ac-8013-f424b67bdb84 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:28.932758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b98df-ab3e-4d8b-82fb-802998f33ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:28.933399 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b98df-ab3e-4d8b-82fb-802998f33ed7 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:28.933683 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b98df-ab3e-4d8b-82fb-802998f33ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:28.934000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b98df-ab3e-4d8b-82fb-802998f33ed7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:28.941965 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b98df-ab3e-4d8b-82fb-802998f33ed7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:28.943401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b98df-ab3e-4d8b-82fb-802998f33ed7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-7f8b98df-ab3e-4d8b-82fb-802998f33ed7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:28.955738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:28.956836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:28.957832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:28.957832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:28.957832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:28.957832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:28.957832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:28.957832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:28.957832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:28.957832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:28.957832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 14:39:28.957832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:28.957832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 296/575] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:28.957832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b98df-ab3e-4d8b-82fb-802998f33ed7 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:28.971417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050f037-4798-4900-bc9f-8438ab00306a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:28.978484 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:29.010500 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebbc0a-e22b-4d3f-8cb3-8d7c7c32cfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:29.011170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebbc0a-e22b-4d3f-8cb3-8d7c7c32cfc3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:29.011418 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebbc0a-e22b-4d3f-8cb3-8d7c7c32cfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:29.011707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebbc0a-e22b-4d3f-8cb3-8d7c7c32cfc3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:29.018735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ebbc0a-e22b-4d3f-8cb3-8d7c7c32cfc3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:29.020235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ebbc0a-e22b-4d3f-8cb3-8d7c7c32cfc3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:29.032280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-32ebbc0a-e22b-4d3f-8cb3-8d7c7c32cfc3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:29.032280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:29.032280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:29.032280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:29.032280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:29.032280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:29.032280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:29.032280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:29.032280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:29.032280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:29.032280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:29.032280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:29.032280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:29.032280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:29.032280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:29.033397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:29.034486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:29.034486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:29.034486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:29.034486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:29.034486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:29.034486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:29.034486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:29.034486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:29.038451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 280/576] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:29.039352 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebbc0a-e22b-4d3f-8cb3-8d7c7c32cfc3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:29.078581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b98df-ab3e-4d8b-82fb-802998f33ed7 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:29.106037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:29.115951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9025668-4c95-4233-a0d0-741837435643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:29.116640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9025668-4c95-4233-a0d0-741837435643 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:29.117112 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9025668-4c95-4233-a0d0-741837435643 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:29.117388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9025668-4c95-4233-a0d0-741837435643 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:29.118825 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9025668-4c95-4233-a0d0-741837435643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:29.119131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9025668-4c95-4233-a0d0-741837435643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:29.120530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9025668-4c95-4233-a0d0-741837435643 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:29.133365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 297/577] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:29.134222 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9025668-4c95-4233-a0d0-741837435643 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:29.151203 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebbc0a-e22b-4d3f-8cb3-8d7c7c32cfc3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:29.186678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb9069-359b-48ec-969f-69937bad4c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:29.187289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb9069-359b-48ec-969f-69937bad4c6a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:29.187574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb9069-359b-48ec-969f-69937bad4c6a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:29.187880 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb9069-359b-48ec-969f-69937bad4c6a None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:29.188522 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb9069-359b-48ec-969f-69937bad4c6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:29.188830 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb9069-359b-48ec-969f-69937bad4c6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:29.190211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bb9069-359b-48ec-969f-69937bad4c6a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:29.203245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 281/578] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:29.204118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb9069-359b-48ec-969f-69937bad4c6a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:29.217493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:29.242885 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9025668-4c95-4233-a0d0-741837435643 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:29.283398 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88212b-6da6-4142-b26c-f8c0eed3ff4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:29.284037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88212b-6da6-4142-b26c-f8c0eed3ff4f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:29.284327 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88212b-6da6-4142-b26c-f8c0eed3ff4f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:29.284594 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88212b-6da6-4142-b26c-f8c0eed3ff4f None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:29.285319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88212b-6da6-4142-b26c-f8c0eed3ff4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:29.285653 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88212b-6da6-4142-b26c-f8c0eed3ff4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:29.287035 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88212b-6da6-4142-b26c-f8c0eed3ff4f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:29.304719 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 298/579] 10.222.0.60 () {56 vars in 1137 bytes} [Mon Sep 19 14:39:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:29.305530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88212b-6da6-4142-b26c-f8c0eed3ff4f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:29.306001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb9069-359b-48ec-969f-69937bad4c6a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:29.328690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:29.343366 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fa8a2-06d8-41bc-a1db-8dc5413f479a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:29.344011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fa8a2-06d8-41bc-a1db-8dc5413f479a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:29.344292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fa8a2-06d8-41bc-a1db-8dc5413f479a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:29.344563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fa8a2-06d8-41bc-a1db-8dc5413f479a None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:29.345246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fa8a2-06d8-41bc-a1db-8dc5413f479a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:29.345552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fa8a2-06d8-41bc-a1db-8dc5413f479a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:29.346994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fa8a2-06d8-41bc-a1db-8dc5413f479a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:29.359100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 282/580] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:29.359955 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fa8a2-06d8-41bc-a1db-8dc5413f479a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:29.412692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88212b-6da6-4142-b26c-f8c0eed3ff4f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:29.442504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:29.447779 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc59807-9b50-49b6-bc2d-92b4d16e0cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:29.448437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc59807-9b50-49b6-bc2d-92b4d16e0cac None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:29.448709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc59807-9b50-49b6-bc2d-92b4d16e0cac None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:29.448975 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc59807-9b50-49b6-bc2d-92b4d16e0cac None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:29.459104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc59807-9b50-49b6-bc2d-92b4d16e0cac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:29.462784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fa8a2-06d8-41bc-a1db-8dc5413f479a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-0fc59807-9b50-49b6-bc2d-92b4d16e0cac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:29.463985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:29.465126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:29.466597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:29.466597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:29.466597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:29.466597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:29.466597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:29.466597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:29.466597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:29.466597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:29.466597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:29.466597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:29.466597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:29.466597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:29.466597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 299/581] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:29.466597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc59807-9b50-49b6-bc2d-92b4d16e0cac None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:29.500492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622fe168-7303-47ac-aabf-7299522491dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:29.501037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fe168-7303-47ac-aabf-7299522491dd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:29.501323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fe168-7303-47ac-aabf-7299522491dd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:29.501588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622fe168-7303-47ac-aabf-7299522491dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:29.508802 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622fe168-7303-47ac-aabf-7299522491dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:29.510278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622fe168-7303-47ac-aabf-7299522491dd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:29.521434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-622fe168-7303-47ac-aabf-7299522491dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:29.521434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:29.521434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:29.521434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:29.521434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:29.521434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:29.521434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:29.521434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:29.521434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:29.521434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:29.521434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:29.521434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:29.521434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:29.521434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:29.521434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:29.522497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:29.523584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:29.523584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:29.523584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:29.523584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:29.523584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:29.523584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:29.523584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:29.523584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:29.523584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 283/582] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:29.523584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622fe168-7303-47ac-aabf-7299522491dd None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:29.565822 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:29.570090 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc59807-9b50-49b6-bc2d-92b4d16e0cac None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:29.603815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e8324-9a84-46cc-b4ad-58158d94a1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:29.604479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e8324-9a84-46cc-b4ad-58158d94a1fd None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:29.605042 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e8324-9a84-46cc-b4ad-58158d94a1fd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:29.605180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e8324-9a84-46cc-b4ad-58158d94a1fd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:29.613819 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e8324-9a84-46cc-b4ad-58158d94a1fd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:29.615272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e8324-9a84-46cc-b4ad-58158d94a1fd None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:29.624896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622fe168-7303-47ac-aabf-7299522491dd None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-fc3e8324-9a84-46cc-b4ad-58158d94a1fd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:29.627573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:29.628720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:29.629856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:29.629856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:29.629856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:29.629856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:29.629856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:29.629856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:29.629856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:29.629856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:29.629856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 14:39:29.629856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:29.632164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 300/583] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:29.632764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e8324-9a84-46cc-b4ad-58158d94a1fd None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:29.658585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89ff49-85b5-42ae-8708-075e0ab5f123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:29.659216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89ff49-85b5-42ae-8708-075e0ab5f123 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:29.659500 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89ff49-85b5-42ae-8708-075e0ab5f123 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:29.659773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89ff49-85b5-42ae-8708-075e0ab5f123 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:29.670148 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89ff49-85b5-42ae-8708-075e0ab5f123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:29.671601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89ff49-85b5-42ae-8708-075e0ab5f123 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:29.685359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-6a89ff49-85b5-42ae-8708-075e0ab5f123 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:29.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:29.687349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:29.688568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:29.688568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:29.688568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:29.688568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 284/584] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:29.688568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89ff49-85b5-42ae-8708-075e0ab5f123 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:29.737976 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e8324-9a84-46cc-b4ad-58158d94a1fd None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:29.771409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9b062-6317-49a1-8e5e-4766ef151b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:29.772019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9b062-6317-49a1-8e5e-4766ef151b02 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:29.772300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9b062-6317-49a1-8e5e-4766ef151b02 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:29.772571 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9b062-6317-49a1-8e5e-4766ef151b02 None admin] PATH_INFO: `/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/59b5d2fd362b417c9b9be75f88a667ad/roles/15fbc4d519f64f4cacecc3b20fe826ae` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:29.782939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:29.787211 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9b062-6317-49a1-8e5e-4766ef151b02 None admin] RBAC: Authorizing `identity:create_grant(project_id=7dee98edf19d478ca4f746ff85579fba, user_id=59b5d2fd362b417c9b9be75f88a667ad, role_id=15fbc4d519f64f4cacecc3b20fe826ae)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:29.788658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9b062-6317-49a1-8e5e-4766ef151b02 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:29.792673 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89ff49-85b5-42ae-8708-075e0ab5f123 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:29.802424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 301/585] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:39:29 2022] PUT /identity/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/59b5d2fd362b417c9b9be75f88a667ad/roles/15fbc4d519f64f4cacecc3b20fe826ae => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:39:29.803227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9b062-6317-49a1-8e5e-4766ef151b02 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:29.832254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6f869-5e4b-4f71-84d2-c0caf1f2aeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:29.832881 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6f869-5e4b-4f71-84d2-c0caf1f2aeae None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:29.833161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6f869-5e4b-4f71-84d2-c0caf1f2aeae None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:29.833431 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6f869-5e4b-4f71-84d2-c0caf1f2aeae None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:29.834116 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6f869-5e4b-4f71-84d2-c0caf1f2aeae None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:29.834424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6f869-5e4b-4f71-84d2-c0caf1f2aeae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:29.836089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b6f869-5e4b-4f71-84d2-c0caf1f2aeae None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:29.845985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 285/586] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:29.846853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6f869-5e4b-4f71-84d2-c0caf1f2aeae None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:29.944873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9b062-6317-49a1-8e5e-4766ef151b02 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:29.946882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:29.967589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b6f869-5e4b-4f71-84d2-c0caf1f2aeae None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:29.991564 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2bce9-993d-4cc8-b78e-fc10f6c10a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:29.992213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2bce9-993d-4cc8-b78e-fc10f6c10a98 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:29.992491 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2bce9-993d-4cc8-b78e-fc10f6c10a98 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:29.992765 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2bce9-993d-4cc8-b78e-fc10f6c10a98 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:29.993454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2bce9-993d-4cc8-b78e-fc10f6c10a98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:29.993759 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2bce9-993d-4cc8-b78e-fc10f6c10a98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:29.995192 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2bce9-993d-4cc8-b78e-fc10f6c10a98 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:30.008638 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 302/587] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:30.009482 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2bce9-993d-4cc8-b78e-fc10f6c10a98 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:30.009609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689db55-4f20-4f3d-b0f9-bcf8fe6a2c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:30.010211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689db55-4f20-4f3d-b0f9-bcf8fe6a2c2e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:30.010493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689db55-4f20-4f3d-b0f9-bcf8fe6a2c2e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:30.010769 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689db55-4f20-4f3d-b0f9-bcf8fe6a2c2e None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:30.011491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689db55-4f20-4f3d-b0f9-bcf8fe6a2c2e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:30.011855 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689db55-4f20-4f3d-b0f9-bcf8fe6a2c2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:30.013891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689db55-4f20-4f3d-b0f9-bcf8fe6a2c2e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:30.032226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 286/588] 10.222.0.60 () {56 vars in 1137 bytes} [Mon Sep 19 14:39:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:30.032877 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689db55-4f20-4f3d-b0f9-bcf8fe6a2c2e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:30.049805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:30.127027 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2bce9-993d-4cc8-b78e-fc10f6c10a98 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:30.145835 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689db55-4f20-4f3d-b0f9-bcf8fe6a2c2e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:30.163671 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ce487-231c-4eaf-80b5-d8dd3e0d5d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:30.164308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ce487-231c-4eaf-80b5-d8dd3e0d5d2f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:30.164592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ce487-231c-4eaf-80b5-d8dd3e0d5d2f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:30.164860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ce487-231c-4eaf-80b5-d8dd3e0d5d2f None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:30.165488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ce487-231c-4eaf-80b5-d8dd3e0d5d2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:30.165786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ce487-231c-4eaf-80b5-d8dd3e0d5d2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:30.167208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ce487-231c-4eaf-80b5-d8dd3e0d5d2f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:30.180003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 303/589] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:30.180834 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ce487-231c-4eaf-80b5-d8dd3e0d5d2f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:30.185981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e977b-00ee-46e3-b0a8-cc76f27222b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:30.186629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e977b-00ee-46e3-b0a8-cc76f27222b7 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:30.186908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e977b-00ee-46e3-b0a8-cc76f27222b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:30.187172 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e977b-00ee-46e3-b0a8-cc76f27222b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:30.198184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e977b-00ee-46e3-b0a8-cc76f27222b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:30.199644 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e977b-00ee-46e3-b0a8-cc76f27222b7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:30.207585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-e79e977b-00ee-46e3-b0a8-cc76f27222b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:30.215751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:30.217557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:30.218500 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:30.218500 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:30.218500 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:30.218500 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 287/590] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:30.218500 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e977b-00ee-46e3-b0a8-cc76f27222b7 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:30.286725 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:30.299353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ce487-231c-4eaf-80b5-d8dd3e0d5d2f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:30.333668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649bf40-9a52-4904-b7eb-3eba95070215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:30.334267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649bf40-9a52-4904-b7eb-3eba95070215 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:30.334386 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e977b-00ee-46e3-b0a8-cc76f27222b7 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:30.334588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649bf40-9a52-4904-b7eb-3eba95070215 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:30.334850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649bf40-9a52-4904-b7eb-3eba95070215 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:30.335509 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649bf40-9a52-4904-b7eb-3eba95070215 None admin] RBAC: Adding query filter params (role.id=15fbc4d519f64f4cacecc3b20fe826ae, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=bfc14d32f4314c6e93d20a9c5a45c220) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:30.335813 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649bf40-9a52-4904-b7eb-3eba95070215 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:30.337248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649bf40-9a52-4904-b7eb-3eba95070215 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:30.343261 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 304/591] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:39:30 2022] GET /identity/v3/role_assignments?user.id=bfc14d32f4314c6e93d20a9c5a45c220&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=15fbc4d519f64f4cacecc3b20fe826ae => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:30.344082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649bf40-9a52-4904-b7eb-3eba95070215 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:30.384703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88aef3-c470-495d-9ff9-ec28352baa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:30.386068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88aef3-c470-495d-9ff9-ec28352baa52 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:30.386989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88aef3-c470-495d-9ff9-ec28352baa52 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:30.387760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88aef3-c470-495d-9ff9-ec28352baa52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:30.404234 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88aef3-c470-495d-9ff9-ec28352baa52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:30.406863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88aef3-c470-495d-9ff9-ec28352baa52 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-fd88aef3-c470-495d-9ff9-ec28352baa52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:30.429612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:30.430672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:30.431554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:30.431554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:30.431554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:30.438813 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 288/592] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:30.438813 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88aef3-c470-495d-9ff9-ec28352baa52 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:30.466806 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:30.483004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649bf40-9a52-4904-b7eb-3eba95070215 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:30.523264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df7177-2253-433c-a434-7e306c0c3bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:30.524448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df7177-2253-433c-a434-7e306c0c3bf2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:30.525000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df7177-2253-433c-a434-7e306c0c3bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:30.525487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df7177-2253-433c-a434-7e306c0c3bf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:30.534260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df7177-2253-433c-a434-7e306c0c3bf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:30.536258 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df7177-2253-433c-a434-7e306c0c3bf2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:30.550835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-d6df7177-2253-433c-a434-7e306c0c3bf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:30.550835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:30.550835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:30.550835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:30.550835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:30.550835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:30.550835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:30.550835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:30.550835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:30.550835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:30.550835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:30.550835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:30.550835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:30.550835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:30.550835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:30.551950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:30.553070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:30.553070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:30.553070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:30.553070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:30.553070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:30.553070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:30.553070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:30.553070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:30.557330 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 305/593] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:30.558213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df7177-2253-433c-a434-7e306c0c3bf2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:30.562301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88aef3-c470-495d-9ff9-ec28352baa52 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:30.580358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:30.605479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bd479-86c5-47a6-8c67-963b7c5dba36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:30.606864 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bd479-86c5-47a6-8c67-963b7c5dba36 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:30.607570 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bd479-86c5-47a6-8c67-963b7c5dba36 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:30.608035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bd479-86c5-47a6-8c67-963b7c5dba36 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:30.608720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bd479-86c5-47a6-8c67-963b7c5dba36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:30.609082 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bd479-86c5-47a6-8c67-963b7c5dba36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:30.611878 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bd479-86c5-47a6-8c67-963b7c5dba36 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:30.633966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 289/594] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:30.633966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bd479-86c5-47a6-8c67-963b7c5dba36 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:30.679930 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df7177-2253-433c-a434-7e306c0c3bf2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:30.724601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3b680-8e80-4436-b1db-6e4bb15e165e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:30.725232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3b680-8e80-4436-b1db-6e4bb15e165e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:30.725515 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3b680-8e80-4436-b1db-6e4bb15e165e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:30.725793 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b3b680-8e80-4436-b1db-6e4bb15e165e None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:30.726449 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3b680-8e80-4436-b1db-6e4bb15e165e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:30.726876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3b680-8e80-4436-b1db-6e4bb15e165e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:30.728418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b3b680-8e80-4436-b1db-6e4bb15e165e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:30.745839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 306/595] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:30.746505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3b680-8e80-4436-b1db-6e4bb15e165e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:30.757954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:30.766831 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bd479-86c5-47a6-8c67-963b7c5dba36 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:30.805019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b928253-d1ba-450e-a1cf-ea5e3be9c95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:30.805975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b928253-d1ba-450e-a1cf-ea5e3be9c95e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:30.806554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b928253-d1ba-450e-a1cf-ea5e3be9c95e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:30.807041 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b928253-d1ba-450e-a1cf-ea5e3be9c95e None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:30.808075 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b928253-d1ba-450e-a1cf-ea5e3be9c95e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:30.808578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b928253-d1ba-450e-a1cf-ea5e3be9c95e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:30.810363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b928253-d1ba-450e-a1cf-ea5e3be9c95e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:30.823520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 290/596] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:30.824852 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b928253-d1ba-450e-a1cf-ea5e3be9c95e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:30.866743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:30.871501 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b3b680-8e80-4436-b1db-6e4bb15e165e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:30.918424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f97bdd-bd37-44b8-88e1-2bd2be111c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:30.919978 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f97bdd-bd37-44b8-88e1-2bd2be111c01 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:30.920410 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f97bdd-bd37-44b8-88e1-2bd2be111c01 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:30.920846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f97bdd-bd37-44b8-88e1-2bd2be111c01 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:30.930806 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f97bdd-bd37-44b8-88e1-2bd2be111c01 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:30.932727 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f97bdd-bd37-44b8-88e1-2bd2be111c01 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-f6f97bdd-bd37-44b8-88e1-2bd2be111c01 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:30.949364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:30.950993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:30.951986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:30.951986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:30.951986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:30.951986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:30.951986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:30.951986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:30.951986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:30.951986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:30.951986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 14:39:30.951986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:30.953211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b928253-d1ba-450e-a1cf-ea5e3be9c95e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:30.954917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 307/597] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:30.956036 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f97bdd-bd37-44b8-88e1-2bd2be111c01 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:31.006103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c28059-00cc-4906-8281-d7c8bd34c322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:31.006856 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c28059-00cc-4906-8281-d7c8bd34c322 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:31.007199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c28059-00cc-4906-8281-d7c8bd34c322 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:31.007538 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c28059-00cc-4906-8281-d7c8bd34c322 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:31.017669 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c28059-00cc-4906-8281-d7c8bd34c322 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:31.019453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c28059-00cc-4906-8281-d7c8bd34c322 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:31.035314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c7c28059-00cc-4906-8281-d7c8bd34c322 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:31.035314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:31.035314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:31.035314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:31.035314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:31.035314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:31.035314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:31.035314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:31.035314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:31.035314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:31.035314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:31.035314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:31.035314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:31.035314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:31.035314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:31.036377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:31.037297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:31.037297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:31.037297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:31.037297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:31.037297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:31.037297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:31.037297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:31.037297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:31.037297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 291/598] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:31.037790 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c28059-00cc-4906-8281-d7c8bd34c322 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:31.040914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:31.085410 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f97bdd-bd37-44b8-88e1-2bd2be111c01 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:31.134501 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126668db-fe8c-42d4-9fd5-2ad698b6a8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:31.135841 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126668db-fe8c-42d4-9fd5-2ad698b6a8a7 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:31.136355 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126668db-fe8c-42d4-9fd5-2ad698b6a8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:31.137041 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126668db-fe8c-42d4-9fd5-2ad698b6a8a7 None admin] PATH_INFO: `/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/caab7aaa72634b358d717768fbcb48d8/roles/15fbc4d519f64f4cacecc3b20fe826ae` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:31.158267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126668db-fe8c-42d4-9fd5-2ad698b6a8a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=7dee98edf19d478ca4f746ff85579fba, user_id=caab7aaa72634b358d717768fbcb48d8, role_id=15fbc4d519f64f4cacecc3b20fe826ae)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:31.160639 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126668db-fe8c-42d4-9fd5-2ad698b6a8a7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:31.164912 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:31.173173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c28059-00cc-4906-8281-d7c8bd34c322 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:31.180135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 308/599] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:39:30 2022] PUT /identity/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/caab7aaa72634b358d717768fbcb48d8/roles/15fbc4d519f64f4cacecc3b20fe826ae => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:39:31.181326 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126668db-fe8c-42d4-9fd5-2ad698b6a8a7 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:31.258980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c0bdd-cfe4-4258-9ab8-7c4f3af1ba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:31.259986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c0bdd-cfe4-4258-9ab8-7c4f3af1ba13 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:31.260459 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c0bdd-cfe4-4258-9ab8-7c4f3af1ba13 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:31.260918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c0bdd-cfe4-4258-9ab8-7c4f3af1ba13 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:31.262087 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c0bdd-cfe4-4258-9ab8-7c4f3af1ba13 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:31.263802 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c0bdd-cfe4-4258-9ab8-7c4f3af1ba13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:31.265907 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c0bdd-cfe4-4258-9ab8-7c4f3af1ba13 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:31.304251 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 292/600] 10.222.0.60 () {56 vars in 1139 bytes} [Mon Sep 19 14:39:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:31.313051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c0bdd-cfe4-4258-9ab8-7c4f3af1ba13 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:31.412397 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126668db-fe8c-42d4-9fd5-2ad698b6a8a7 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:31.438559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:31.463043 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df16902-9f18-4980-adcf-080574f8bae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:31.463969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df16902-9f18-4980-adcf-080574f8bae2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:31.464464 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df16902-9f18-4980-adcf-080574f8bae2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:31.465186 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df16902-9f18-4980-adcf-080574f8bae2 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:31.467580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df16902-9f18-4980-adcf-080574f8bae2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:31.468551 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df16902-9f18-4980-adcf-080574f8bae2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:31.470709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df16902-9f18-4980-adcf-080574f8bae2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:31.478208 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c0bdd-cfe4-4258-9ab8-7c4f3af1ba13 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:31.487928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 309/601] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:31.498231 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df16902-9f18-4980-adcf-080574f8bae2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:31.532034 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0585a9-7819-4c64-b06b-67fa3e0e4896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:31.532680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0585a9-7819-4c64-b06b-67fa3e0e4896 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:31.532961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0585a9-7819-4c64-b06b-67fa3e0e4896 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:31.533233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0585a9-7819-4c64-b06b-67fa3e0e4896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:31.553199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0585a9-7819-4c64-b06b-67fa3e0e4896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:31.554820 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0585a9-7819-4c64-b06b-67fa3e0e4896 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-4a0585a9-7819-4c64-b06b-67fa3e0e4896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:31.580632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:31.582084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:31.582981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:31.582981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:31.582981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:31.588485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 293/602] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:39:31.594817 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0585a9-7819-4c64-b06b-67fa3e0e4896 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:31.612939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:31.624756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df16902-9f18-4980-adcf-080574f8bae2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:31.661969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536f438-a42a-4fc9-9761-a6e44e90f6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:31.663071 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536f438-a42a-4fc9-9761-a6e44e90f6c5 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:31.663555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536f438-a42a-4fc9-9761-a6e44e90f6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:31.664055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536f438-a42a-4fc9-9761-a6e44e90f6c5 None admin] PATH_INFO: `/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/006fe9ba3b0a40f29fa2be7620e548c4/roles/15fbc4d519f64f4cacecc3b20fe826ae` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:31.681229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536f438-a42a-4fc9-9761-a6e44e90f6c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7dee98edf19d478ca4f746ff85579fba, user_id=006fe9ba3b0a40f29fa2be7620e548c4, role_id=15fbc4d519f64f4cacecc3b20fe826ae)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:31.683618 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536f438-a42a-4fc9-9761-a6e44e90f6c5 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:31.702138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 310/603] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:39:31 2022] PUT /identity/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/006fe9ba3b0a40f29fa2be7620e548c4/roles/15fbc4d519f64f4cacecc3b20fe826ae => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:39:31.802471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0585a9-7819-4c64-b06b-67fa3e0e4896 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:31.839536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63a716-9665-4758-9c62-6c693bd90fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:31.840188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63a716-9665-4758-9c62-6c693bd90fc3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:31.840464 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63a716-9665-4758-9c62-6c693bd90fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:31.840737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63a716-9665-4758-9c62-6c693bd90fc3 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:31.841908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63a716-9665-4758-9c62-6c693bd90fc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:31.842211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63a716-9665-4758-9c62-6c693bd90fc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:31.843619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63a716-9665-4758-9c62-6c693bd90fc3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:31.862055 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 294/604] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:39:31.863577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536f438-a42a-4fc9-9761-a6e44e90f6c5 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:31.906743 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:32.011411 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536f438-a42a-4fc9-9761-a6e44e90f6c5 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:32.054819 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659687cc-b96a-402e-b4c6-3f026b4a842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:32.055464 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659687cc-b96a-402e-b4c6-3f026b4a842d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:32.055742 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659687cc-b96a-402e-b4c6-3f026b4a842d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:32.056045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659687cc-b96a-402e-b4c6-3f026b4a842d None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:32.063239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659687cc-b96a-402e-b4c6-3f026b4a842d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:32.064763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659687cc-b96a-402e-b4c6-3f026b4a842d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:32.077559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-659687cc-b96a-402e-b4c6-3f026b4a842d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:32.077559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:32.077559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:32.077559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:32.077559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:32.077559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:32.077559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:32.077559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:32.077559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:32.077559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:32.077559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:32.077559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:32.077559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:32.077559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:32.077559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:32.078505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:32.079419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:32.079419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:32.079419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:32.079419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:32.079419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:32.079419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:32.079419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:32.079419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:32.083131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 311/605] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:32.088596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63a716-9665-4758-9c62-6c693bd90fc3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:32.162840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:32.170127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698841d-ef15-4acc-976e-fce6cc1b0e5c None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:32.170408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698841d-ef15-4acc-976e-fce6cc1b0e5c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:32.170680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698841d-ef15-4acc-976e-fce6cc1b0e5c None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:32.172231 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 312/606] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:39:32.183181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170707c-5e62-447e-83e9-6f7546d5286a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:32.183462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170707c-5e62-447e-83e9-6f7546d5286a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:32.183736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170707c-5e62-447e-83e9-6f7546d5286a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:32.211962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2170707c-5e62-447e-83e9-6f7546d5286a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:32.213206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2170707c-5e62-447e-83e9-6f7546d5286a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:32.243523 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63a716-9665-4758-9c62-6c693bd90fc3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:32.291303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d08e48-a4f6-4d0d-9303-09dd74375909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:32.291849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d08e48-a4f6-4d0d-9303-09dd74375909 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:32.292214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d08e48-a4f6-4d0d-9303-09dd74375909 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:32.292454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d08e48-a4f6-4d0d-9303-09dd74375909 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:32.293148 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d08e48-a4f6-4d0d-9303-09dd74375909 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:32.293447 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d08e48-a4f6-4d0d-9303-09dd74375909 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:32.294950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d08e48-a4f6-4d0d-9303-09dd74375909 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:32.312053 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 295/607] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:32.314197 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d08e48-a4f6-4d0d-9303-09dd74375909 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:32.366240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 313/608] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:39:32.446538 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d08e48-a4f6-4d0d-9303-09dd74375909 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:32.481723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:32.493433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d4ad10-7692-410f-acb8-8280c1029d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:32.494091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4ad10-7692-410f-acb8-8280c1029d3a None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:32.494375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4ad10-7692-410f-acb8-8280c1029d3a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:32.494646 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d4ad10-7692-410f-acb8-8280c1029d3a None admin] PATH_INFO: `/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/2fbef7b24f134f5eb1193aed1b7ee69c/roles/15fbc4d519f64f4cacecc3b20fe826ae` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:32.513382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d4ad10-7692-410f-acb8-8280c1029d3a None admin] RBAC: Authorizing `identity:create_grant(project_id=7dee98edf19d478ca4f746ff85579fba, user_id=2fbef7b24f134f5eb1193aed1b7ee69c, role_id=15fbc4d519f64f4cacecc3b20fe826ae)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:32.514830 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d4ad10-7692-410f-acb8-8280c1029d3a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:32.532897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 296/609] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:39:32 2022] PUT /identity/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/2fbef7b24f134f5eb1193aed1b7ee69c/roles/15fbc4d519f64f4cacecc3b20fe826ae => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:39:32.653133 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170707c-5e62-447e-83e9-6f7546d5286a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:32.672913 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:32.748433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e1556-2db7-4b67-b93f-1ce4e38c4e88 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:32.748964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e1556-2db7-4b67-b93f-1ce4e38c4e88 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:32.748964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e1556-2db7-4b67-b93f-1ce4e38c4e88 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:32.750425 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 297/610] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:32.759609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9af44-100e-4457-a60b-dc6c51ec2f7f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:32.759609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9af44-100e-4457-a60b-dc6c51ec2f7f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:32.759609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9af44-100e-4457-a60b-dc6c51ec2f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:32.764629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170707c-5e62-447e-83e9-6f7546d5286a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:32.806951 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-efc9af44-100e-4457-a60b-dc6c51ec2f7f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:32.810193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-efc9af44-100e-4457-a60b-dc6c51ec2f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:32.812413 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707b1cd-fadc-44dd-b2ab-02783d68e23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:32.813656 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707b1cd-fadc-44dd-b2ab-02783d68e23a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:32.814357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707b1cd-fadc-44dd-b2ab-02783d68e23a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:32.814800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707b1cd-fadc-44dd-b2ab-02783d68e23a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:32.836209 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707b1cd-fadc-44dd-b2ab-02783d68e23a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-4707b1cd-fadc-44dd-b2ab-02783d68e23a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:32.842563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:32.843666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:32.844577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:32.844577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:32.844577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:32.844577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:32.844577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:32.844577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:32.844577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:32.844577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:32.844577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:32.844577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:32.844577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:32.844577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:32.847188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 314/611] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:32.853101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707b1cd-fadc-44dd-b2ab-02783d68e23a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:32.910950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 298/612] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 19 14:39:32.941900 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:32.968264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707b1cd-fadc-44dd-b2ab-02783d68e23a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:33.003737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08829dd6-7345-464c-9b78-e3f632c2c43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:33.004389 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08829dd6-7345-464c-9b78-e3f632c2c43d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:33.004667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08829dd6-7345-464c-9b78-e3f632c2c43d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:33.004939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08829dd6-7345-464c-9b78-e3f632c2c43d None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:33.005597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08829dd6-7345-464c-9b78-e3f632c2c43d None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:33.005897 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08829dd6-7345-464c-9b78-e3f632c2c43d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:33.007264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08829dd6-7345-464c-9b78-e3f632c2c43d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:33.016769 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 315/613] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:39:33.022471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9af44-100e-4457-a60b-dc6c51ec2f7f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:33.178311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9af44-100e-4457-a60b-dc6c51ec2f7f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:33.187142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:33.228667 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023e5a8-820c-4b59-8296-6f7082439c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:33.229302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023e5a8-820c-4b59-8296-6f7082439c9d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:33.229583 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023e5a8-820c-4b59-8296-6f7082439c9d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:33.229858 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023e5a8-820c-4b59-8296-6f7082439c9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:33.245283 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08829dd6-7345-464c-9b78-e3f632c2c43d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:33.250787 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023e5a8-820c-4b59-8296-6f7082439c9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:33.252275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023e5a8-820c-4b59-8296-6f7082439c9d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-3023e5a8-820c-4b59-8296-6f7082439c9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:33.276370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:33.277399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:33.278317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:33.278317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:33.278317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:33.278937 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 299/614] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:33.284581 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023e5a8-820c-4b59-8296-6f7082439c9d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:33.419991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08829dd6-7345-464c-9b78-e3f632c2c43d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:33.482538 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023e5a8-820c-4b59-8296-6f7082439c9d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:33.485464 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030b974-4a17-4e48-8600-071ab0cc8fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:33.486107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030b974-4a17-4e48-8600-071ab0cc8fe8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:33.486384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030b974-4a17-4e48-8600-071ab0cc8fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:33.486657 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b030b974-4a17-4e48-8600-071ab0cc8fe8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:33.501561 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:33.507808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b030b974-4a17-4e48-8600-071ab0cc8fe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-b030b974-4a17-4e48-8600-071ab0cc8fe8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:33.513510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:33.515667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:33.517744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:33.517744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:33.517744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:33.517744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:33.517744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:33.517744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:33.517744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:33.517744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:33.517744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:33.517744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:33.517744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:33.517744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:33.517744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 316/615] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:39:33.523908 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030b974-4a17-4e48-8600-071ab0cc8fe8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:33.527544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a96e03-1f6e-45ee-b9b0-f80bbf090d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:33.528196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a96e03-1f6e-45ee-b9b0-f80bbf090d56 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:33.528545 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a96e03-1f6e-45ee-b9b0-f80bbf090d56 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:33.529310 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a96e03-1f6e-45ee-b9b0-f80bbf090d56 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:33.529958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a96e03-1f6e-45ee-b9b0-f80bbf090d56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:33.530259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a96e03-1f6e-45ee-b9b0-f80bbf090d56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:33.532617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a96e03-1f6e-45ee-b9b0-f80bbf090d56 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:33.558049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 300/616] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:33.563732 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a96e03-1f6e-45ee-b9b0-f80bbf090d56 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:33.596048 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:33.673078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a96e03-1f6e-45ee-b9b0-f80bbf090d56 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:33.711657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e868d0-ceaa-49ec-a484-bd1b375a0052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:33.716564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e868d0-ceaa-49ec-a484-bd1b375a0052 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:33.716854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e868d0-ceaa-49ec-a484-bd1b375a0052 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:33.717139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e868d0-ceaa-49ec-a484-bd1b375a0052 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:33.744155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b030b974-4a17-4e48-8600-071ab0cc8fe8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:33.744682 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e868d0-ceaa-49ec-a484-bd1b375a0052 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:33.746225 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e868d0-ceaa-49ec-a484-bd1b375a0052 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-a2e868d0-ceaa-49ec-a484-bd1b375a0052 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:33.767120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:33.768607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:33.769544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:33.769544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:33.769544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:33.769544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:33.769544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:33.769544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:33.769544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:33.769544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:33.769544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:39:33.769544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:33.772271 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 301/617] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Sep 19 14:39:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:33.787964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e868d0-ceaa-49ec-a484-bd1b375a0052 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:33.804085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583b31e-60af-4939-923e-91252e1f9f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:33.804588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583b31e-60af-4939-923e-91252e1f9f70 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:33.804588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583b31e-60af-4939-923e-91252e1f9f70 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:33.804881 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583b31e-60af-4939-923e-91252e1f9f70 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:33.807852 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583b31e-60af-4939-923e-91252e1f9f70 None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:33.807852 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583b31e-60af-4939-923e-91252e1f9f70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:33.807852 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583b31e-60af-4939-923e-91252e1f9f70 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:33.827539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 317/618] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:33.833879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583b31e-60af-4939-923e-91252e1f9f70 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:33.905373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:33.946663 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583b31e-60af-4939-923e-91252e1f9f70 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:33.982226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:33.983853 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585e703-1cd9-41c3-a833-a168d74e673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:33.984738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585e703-1cd9-41c3-a833-a168d74e673a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:33.985033 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585e703-1cd9-41c3-a833-a168d74e673a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:33.985498 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585e703-1cd9-41c3-a833-a168d74e673a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:33.996914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585e703-1cd9-41c3-a833-a168d74e673a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:33.998392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f585e703-1cd9-41c3-a833-a168d74e673a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-f585e703-1cd9-41c3-a833-a168d74e673a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:34.014637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:34.015871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:34.016786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:34.016786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:34.016786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:34.016786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 318/619] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:34.022318 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585e703-1cd9-41c3-a833-a168d74e673a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:34.045740 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e868d0-ceaa-49ec-a484-bd1b375a0052 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:34.117374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace815dc-94c8-4ee1-b945-59610e0c2e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:34.117997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace815dc-94c8-4ee1-b945-59610e0c2e86 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:34.118279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace815dc-94c8-4ee1-b945-59610e0c2e86 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:34.118555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace815dc-94c8-4ee1-b945-59610e0c2e86 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:34.119288 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace815dc-94c8-4ee1-b945-59610e0c2e86 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:34.119590 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace815dc-94c8-4ee1-b945-59610e0c2e86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:34.121060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace815dc-94c8-4ee1-b945-59610e0c2e86 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:34.155383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 302/620] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Sep 19 14:39:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:39:34.156721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f93bd4-a2b9-44ae-bfb6-aacf4c57635d None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:34.157009 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f93bd4-a2b9-44ae-bfb6-aacf4c57635d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:34.159941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f93bd4-a2b9-44ae-bfb6-aacf4c57635d None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:34.161579 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 303/621] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:34.168215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f93bd4-a2b9-44ae-bfb6-aacf4c57635d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:34.199257 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f585e703-1cd9-41c3-a833-a168d74e673a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:34.219530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:34.243878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e5afe-5a23-4fb8-bb3c-9e6ead668416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:34.244658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e5afe-5a23-4fb8-bb3c-9e6ead668416 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:34.245054 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e5afe-5a23-4fb8-bb3c-9e6ead668416 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:34.245425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e5afe-5a23-4fb8-bb3c-9e6ead668416 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:34.246338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e5afe-5a23-4fb8-bb3c-9e6ead668416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:34.246736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e5afe-5a23-4fb8-bb3c-9e6ead668416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:34.249119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e5afe-5a23-4fb8-bb3c-9e6ead668416 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:34.261804 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 319/622] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:34.268539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86933133-318b-41c8-9764-37a24c5fa538 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:34.268940 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86933133-318b-41c8-9764-37a24c5fa538 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:34.269589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86933133-318b-41c8-9764-37a24c5fa538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:34.289767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-86933133-318b-41c8-9764-37a24c5fa538 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:34.290845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f93bd4-a2b9-44ae-bfb6-aacf4c57635d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:34.291220 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-86933133-318b-41c8-9764-37a24c5fa538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:34.332796 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798d86f-41a4-4942-bae9-24cf8e376843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:34.333562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798d86f-41a4-4942-bae9-24cf8e376843 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:34.333944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798d86f-41a4-4942-bae9-24cf8e376843 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:34.334318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798d86f-41a4-4942-bae9-24cf8e376843 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:34.338649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:34.370914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 320/623] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:34.372860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86933133-318b-41c8-9764-37a24c5fa538 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:34.426470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798d86f-41a4-4942-bae9-24cf8e376843 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:34.428003 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798d86f-41a4-4942-bae9-24cf8e376843 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:34.499237 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86933133-318b-41c8-9764-37a24c5fa538 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:34.501432 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:34.540305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 304/624] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:34.541997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798d86f-41a4-4942-bae9-24cf8e376843 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:34.553815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13e910-eb66-4b0c-84af-35c71d54fcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:34.555258 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13e910-eb66-4b0c-84af-35c71d54fcaf None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:34.555796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13e910-eb66-4b0c-84af-35c71d54fcaf None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:34.556655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13e910-eb66-4b0c-84af-35c71d54fcaf None admin] PATH_INFO: `/v3/users/swift` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:34.568052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d13e910-eb66-4b0c-84af-35c71d54fcaf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:34.570154 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d13e910-eb66-4b0c-84af-35c71d54fcaf None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:34.582674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-7d13e910-eb66-4b0c-84af-35c71d54fcaf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:34.588135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:34.589101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:34.590066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:34.590066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:34.590066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:34.590066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:34.590066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:34.590066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:34.590066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:34.590066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:34.590066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 14:39:34.590066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:34.593880 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 321/625] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:39:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:34.599799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13e910-eb66-4b0c-84af-35c71d54fcaf None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:34.676695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798d86f-41a4-4942-bae9-24cf8e376843 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:34.735851 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5f27e-01a7-4fc2-8a82-7f1f67727ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:34.735851 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5f27e-01a7-4fc2-8a82-7f1f67727ddd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:34.735851 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5f27e-01a7-4fc2-8a82-7f1f67727ddd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:34.735851 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5f27e-01a7-4fc2-8a82-7f1f67727ddd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:34.747852 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13e910-eb66-4b0c-84af-35c71d54fcaf None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:34.756331 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5f27e-01a7-4fc2-8a82-7f1f67727ddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:34.759852 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5f27e-01a7-4fc2-8a82-7f1f67727ddd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-f2b5f27e-01a7-4fc2-8a82-7f1f67727ddd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:34.783874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:34.785337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:34.786262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:34.786262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:34.786262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:34.795795 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 305/626] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:39:34.797220 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb404f-3f17-47a2-a8c0-9a16f6fffcd6 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:34.797506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb404f-3f17-47a2-a8c0-9a16f6fffcd6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:34.797775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb404f-3f17-47a2-a8c0-9a16f6fffcd6 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:34.799574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 306/627] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:34.800049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb404f-3f17-47a2-a8c0-9a16f6fffcd6 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:34.824125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974823e-715d-41db-808e-5efc1d946fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:34.824911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974823e-715d-41db-808e-5efc1d946fa0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:34.825299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974823e-715d-41db-808e-5efc1d946fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:34.825567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974823e-715d-41db-808e-5efc1d946fa0 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:34.826305 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974823e-715d-41db-808e-5efc1d946fa0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:34.826622 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974823e-715d-41db-808e-5efc1d946fa0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:34.828073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974823e-715d-41db-808e-5efc1d946fa0 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:34.855507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 322/628] 10.222.0.60 () {56 vars in 1135 bytes} [Mon Sep 19 14:39:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:39:34.860062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974823e-715d-41db-808e-5efc1d946fa0 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:34.888687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:34.933825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:34.949423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb404f-3f17-47a2-a8c0-9a16f6fffcd6 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:34.987529 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2a77f-2676-4512-ade9-b968299fa97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:34.988490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2a77f-2676-4512-ade9-b968299fa97e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:34.988796 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2a77f-2676-4512-ade9-b968299fa97e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:34.989071 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2a77f-2676-4512-ade9-b968299fa97e None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:34.989709 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea2a77f-2676-4512-ade9-b968299fa97e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:34.990020 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea2a77f-2676-4512-ade9-b968299fa97e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:34.991410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea2a77f-2676-4512-ade9-b968299fa97e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:35.006472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974823e-715d-41db-808e-5efc1d946fa0 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:35.008472 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 307/629] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:35.009780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be73510-9cda-49ed-a865-204d18da2cc0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:35.010070 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be73510-9cda-49ed-a865-204d18da2cc0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:35.010349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be73510-9cda-49ed-a865-204d18da2cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:35.031520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3be73510-9cda-49ed-a865-204d18da2cc0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:35.033120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3be73510-9cda-49ed-a865-204d18da2cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:35.053569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f007fac-6c9f-4b1f-9ec5-40eebbdff533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:35.054208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f007fac-6c9f-4b1f-9ec5-40eebbdff533 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:35.054512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f007fac-6c9f-4b1f-9ec5-40eebbdff533 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:35.054761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f007fac-6c9f-4b1f-9ec5-40eebbdff533 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:35.071446 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f007fac-6c9f-4b1f-9ec5-40eebbdff533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-6f007fac-6c9f-4b1f-9ec5-40eebbdff533 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:35.078590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:35.079589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:35.080486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:35.080486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:35.080486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:35.080486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:35.080486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:35.080486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:35.080486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:35.080486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:35.080486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:35.080486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:35.080486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:35.080486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:35.085514 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 323/630] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:35.086530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f007fac-6c9f-4b1f-9ec5-40eebbdff533 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:35.130972 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 308/631] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:35.132101 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be73510-9cda-49ed-a865-204d18da2cc0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:35.186834 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:35.214988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f007fac-6c9f-4b1f-9ec5-40eebbdff533 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:35.238547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:35.257712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db898b-70e4-4b9f-97c2-271828207f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:35.258573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db898b-70e4-4b9f-97c2-271828207f10 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:35.259099 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db898b-70e4-4b9f-97c2-271828207f10 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:35.259511 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db898b-70e4-4b9f-97c2-271828207f10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:35.266429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be73510-9cda-49ed-a865-204d18da2cc0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:35.271660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db898b-70e4-4b9f-97c2-271828207f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:35.273294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db898b-70e4-4b9f-97c2-271828207f10 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-e0db898b-70e4-4b9f-97c2-271828207f10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:35.290135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:35.291182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:35.292253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:35.292253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:35.292253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:35.294603 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 324/632] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:35.295669 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db898b-70e4-4b9f-97c2-271828207f10 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:35.316040 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a93e73-0429-430f-9568-8289878ab187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:35.316687 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a93e73-0429-430f-9568-8289878ab187 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:35.317233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a93e73-0429-430f-9568-8289878ab187 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:35.317513 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a93e73-0429-430f-9568-8289878ab187 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:35.327317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a93e73-0429-430f-9568-8289878ab187 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:35.329482 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a93e73-0429-430f-9568-8289878ab187 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:35.344718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-33a93e73-0429-430f-9568-8289878ab187 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:35.344718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:35.344718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:35.344718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:35.344718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:35.344718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:35.344718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:35.344718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:35.344718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:35.344718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:35.344718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:35.344718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:35.344718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:35.344718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:35.344718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:35.345672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:35.346589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:35.346589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:35.346589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:35.346589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:35.346589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:35.346589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:35.346589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:35.346589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:35.347880 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 309/633] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:35.348205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a93e73-0429-430f-9568-8289878ab187 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:35.407437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db898b-70e4-4b9f-97c2-271828207f10 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:35.438410 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:35.444142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4d915-a7f6-4e2d-a303-9d9489ae5c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:35.444752 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4d915-a7f6-4e2d-a303-9d9489ae5c36 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:35.445035 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4d915-a7f6-4e2d-a303-9d9489ae5c36 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:35.445306 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4d915-a7f6-4e2d-a303-9d9489ae5c36 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:35.445990 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4d915-a7f6-4e2d-a303-9d9489ae5c36 None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:35.446294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4d915-a7f6-4e2d-a303-9d9489ae5c36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:35.447641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4d915-a7f6-4e2d-a303-9d9489ae5c36 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:35.454235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a93e73-0429-430f-9568-8289878ab187 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:35.456364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 325/634] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:35.457538 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c54188-8673-4861-ac00-4f4ac167b83a None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:35.457828 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c54188-8673-4861-ac00-4f4ac167b83a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:35.458107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c54188-8673-4861-ac00-4f4ac167b83a None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:35.459288 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 326/635] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:35.460004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c54188-8673-4861-ac00-4f4ac167b83a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:35.492119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7b4b7-2e5a-4261-9822-42854ad1f157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:35.492723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7b4b7-2e5a-4261-9822-42854ad1f157 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:35.493000 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7b4b7-2e5a-4261-9822-42854ad1f157 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:35.493266 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7b4b7-2e5a-4261-9822-42854ad1f157 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:35.493897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7b4b7-2e5a-4261-9822-42854ad1f157 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:35.494194 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7b4b7-2e5a-4261-9822-42854ad1f157 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:35.495550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7b4b7-2e5a-4261-9822-42854ad1f157 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:35.502895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:35.508559 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 310/636] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:35.509338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7b4b7-2e5a-4261-9822-42854ad1f157 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:35.562450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c54188-8673-4861-ac00-4f4ac167b83a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:35.596695 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a80f8-345d-4246-9166-ed6962f67b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:35.597306 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a80f8-345d-4246-9166-ed6962f67b47 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:35.597583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a80f8-345d-4246-9166-ed6962f67b47 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:35.597855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a80f8-345d-4246-9166-ed6962f67b47 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:35.598535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a80f8-345d-4246-9166-ed6962f67b47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:35.598838 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a80f8-345d-4246-9166-ed6962f67b47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:35.600292 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a80f8-345d-4246-9166-ed6962f67b47 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:35.609024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7b4b7-2e5a-4261-9822-42854ad1f157 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:35.613659 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 327/637] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:35.614467 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a80f8-345d-4246-9166-ed6962f67b47 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:35.646839 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbdbdc-ab1a-4900-9068-c0712e77b2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:35.647460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbdbdc-ab1a-4900-9068-c0712e77b2ce None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:35.647739 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbdbdc-ab1a-4900-9068-c0712e77b2ce None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:35.648040 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbdbdc-ab1a-4900-9068-c0712e77b2ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:35.658089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbdbdc-ab1a-4900-9068-c0712e77b2ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c6dbdbdc-ab1a-4900-9068-c0712e77b2ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:35.662859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:35.663865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:35.664753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:35.664753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:35.664753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:35.664753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:35.664753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:35.664753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:35.664753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:35.664753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:35.664753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:35.664753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:35.664753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:35.664753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:35.664753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 311/638] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:35.665826 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c4678-9a0c-445b-9dc3-20a0b139514c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:35.665826 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c4678-9a0c-445b-9dc3-20a0b139514c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:35.665826 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c4678-9a0c-445b-9dc3-20a0b139514c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:35.671102 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:35.683377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d70c4678-9a0c-445b-9dc3-20a0b139514c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:35.684604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d70c4678-9a0c-445b-9dc3-20a0b139514c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:35.716604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a80f8-345d-4246-9166-ed6962f67b47 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:35.738975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:35.750843 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ca4b8-e1d1-49cc-8742-8c6edafe703a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:35.751475 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ca4b8-e1d1-49cc-8742-8c6edafe703a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:35.751757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ca4b8-e1d1-49cc-8742-8c6edafe703a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:35.752062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ca4b8-e1d1-49cc-8742-8c6edafe703a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:35.753938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 312/639] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:35.754756 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c4678-9a0c-445b-9dc3-20a0b139514c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:35.762579 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ca4b8-e1d1-49cc-8742-8c6edafe703a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:35.764085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ca4b8-e1d1-49cc-8742-8c6edafe703a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-669ca4b8-e1d1-49cc-8742-8c6edafe703a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:35.779694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:35.780748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:35.781648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:35.781648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:35.781648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:35.781648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 328/640] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:35.783153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ca4b8-e1d1-49cc-8742-8c6edafe703a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:35.869959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c4678-9a0c-445b-9dc3-20a0b139514c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:35.900812 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ca4b8-e1d1-49cc-8742-8c6edafe703a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:35.907792 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:35.911744 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b890c-27fe-4f25-bdcb-7c4061cfa358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:35.912421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b890c-27fe-4f25-bdcb-7c4061cfa358 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:35.913026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b890c-27fe-4f25-bdcb-7c4061cfa358 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:35.913304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b890c-27fe-4f25-bdcb-7c4061cfa358 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:35.920803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b890c-27fe-4f25-bdcb-7c4061cfa358 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:35.922243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b890c-27fe-4f25-bdcb-7c4061cfa358 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:35.934423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-1d3b890c-27fe-4f25-bdcb-7c4061cfa358 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:35.934423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:35.934423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:35.934423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:35.934423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:35.934423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:35.934423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:35.934423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:35.934423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:35.934423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:35.934423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:35.934423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:35.934423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:35.934423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:35.934423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:35.936205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:35.937722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:35.937722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:35.937722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:35.937722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:35.937722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:35.937722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:35.937722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:35.937722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:35.940886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 313/641] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:35.941648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b890c-27fe-4f25-bdcb-7c4061cfa358 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:35.942050 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123627f-df42-4dca-baac-fb0c10c87a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:35.942696 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123627f-df42-4dca-baac-fb0c10c87a34 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:35.943056 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123627f-df42-4dca-baac-fb0c10c87a34 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:35.943398 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123627f-df42-4dca-baac-fb0c10c87a34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:35.944265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123627f-df42-4dca-baac-fb0c10c87a34 None admin] RBAC: Adding query filter params (role.id=15fbc4d519f64f4cacecc3b20fe826ae, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=59b5d2fd362b417c9b9be75f88a667ad) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:35.944642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123627f-df42-4dca-baac-fb0c10c87a34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:35.946166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123627f-df42-4dca-baac-fb0c10c87a34 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:35.953698 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 329/642] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:39:35 2022] GET /identity/v3/role_assignments?user.id=59b5d2fd362b417c9b9be75f88a667ad&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=15fbc4d519f64f4cacecc3b20fe826ae => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:35.959618 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123627f-df42-4dca-baac-fb0c10c87a34 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:36.023984 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:36.090781 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b890c-27fe-4f25-bdcb-7c4061cfa358 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:36.095484 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123627f-df42-4dca-baac-fb0c10c87a34 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:36.128920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc3bc9-2dd0-46b1-b212-908cdb8cf476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:36.129543 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc3bc9-2dd0-46b1-b212-908cdb8cf476 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:36.129827 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc3bc9-2dd0-46b1-b212-908cdb8cf476 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:36.130101 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc3bc9-2dd0-46b1-b212-908cdb8cf476 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:36.130775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc3bc9-2dd0-46b1-b212-908cdb8cf476 None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:36.131074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc3bc9-2dd0-46b1-b212-908cdb8cf476 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:36.132510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdc3bc9-2dd0-46b1-b212-908cdb8cf476 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:36.136442 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1929ae-3df1-4d0e-8c25-b61922325bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:36.137273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1929ae-3df1-4d0e-8c25-b61922325bfd None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:36.137722 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1929ae-3df1-4d0e-8c25-b61922325bfd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:36.138232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1929ae-3df1-4d0e-8c25-b61922325bfd None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:36.139028 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1929ae-3df1-4d0e-8c25-b61922325bfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:36.139336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1929ae-3df1-4d0e-8c25-b61922325bfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:36.140728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1929ae-3df1-4d0e-8c25-b61922325bfd None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:36.143690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 314/643] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:36.143690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc3bc9-2dd0-46b1-b212-908cdb8cf476 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:36.156272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 330/644] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:36.157273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1929ae-3df1-4d0e-8c25-b61922325bfd None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:36.193883 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:36.281597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc3bc9-2dd0-46b1-b212-908cdb8cf476 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:36.282991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1929ae-3df1-4d0e-8c25-b61922325bfd None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:36.292449 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:36.319385 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706102a7-0dcf-4edf-b1a8-2582553005c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:36.319973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706102a7-0dcf-4edf-b1a8-2582553005c4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:36.320303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706102a7-0dcf-4edf-b1a8-2582553005c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:36.321149 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706102a7-0dcf-4edf-b1a8-2582553005c4 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:36.321492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706102a7-0dcf-4edf-b1a8-2582553005c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:36.321795 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706102a7-0dcf-4edf-b1a8-2582553005c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:36.323030 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26210d5e-570d-4ce5-8f27-f0e856bb276e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:36.323655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706102a7-0dcf-4edf-b1a8-2582553005c4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:36.323889 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26210d5e-570d-4ce5-8f27-f0e856bb276e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:36.324272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26210d5e-570d-4ce5-8f27-f0e856bb276e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:36.324644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26210d5e-570d-4ce5-8f27-f0e856bb276e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:36.343011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 315/645] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:36.343011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706102a7-0dcf-4edf-b1a8-2582553005c4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:36.344025 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26210d5e-570d-4ce5-8f27-f0e856bb276e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-26210d5e-570d-4ce5-8f27-f0e856bb276e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:36.350677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:36.352074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:36.353473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:36.353473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:36.353473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:36.353473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:36.353473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:36.353473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:36.353473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:36.353473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:36.353473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:36.353473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:36.353473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:36.353473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:36.353473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 331/646] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:36.355161 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26210d5e-570d-4ce5-8f27-f0e856bb276e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:36.452162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:36.455380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706102a7-0dcf-4edf-b1a8-2582553005c4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:36.462993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26210d5e-570d-4ce5-8f27-f0e856bb276e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:36.496585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158283f-8c0b-4e02-8f4e-933ebeb28c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:36.497280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158283f-8c0b-4e02-8f4e-933ebeb28c84 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:36.497562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158283f-8c0b-4e02-8f4e-933ebeb28c84 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:36.497894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158283f-8c0b-4e02-8f4e-933ebeb28c84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:36.499104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bd56c-478c-4c47-b40a-c41c594e8fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:36.499855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bd56c-478c-4c47-b40a-c41c594e8fa8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:36.500227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bd56c-478c-4c47-b40a-c41c594e8fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:36.500598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bd56c-478c-4c47-b40a-c41c594e8fa8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:36.507843 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7bd56c-478c-4c47-b40a-c41c594e8fa8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:36.508657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158283f-8c0b-4e02-8f4e-933ebeb28c84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:36.509281 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7bd56c-478c-4c47-b40a-c41c594e8fa8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:36.510417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158283f-8c0b-4e02-8f4e-933ebeb28c84 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-1d7bd56c-478c-4c47-b40a-c41c594e8fa8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:36.521387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:36.522549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:36.523502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:36.523502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:36.523502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:36.523502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:36.523502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:36.523502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:36.523502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:36.523502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:36.523502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 19 14:39:36.523502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:36.523502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 332/647] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:36.523502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bd56c-478c-4c47-b40a-c41c594e8fa8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-6158283f-8c0b-4e02-8f4e-933ebeb28c84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:36.531213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:36.532412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:36.533332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:36.533332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:36.533332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:36.533332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 316/648] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:36.533332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158283f-8c0b-4e02-8f4e-933ebeb28c84 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:36.559483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:36.639071 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158283f-8c0b-4e02-8f4e-933ebeb28c84 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:36.643664 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bd56c-478c-4c47-b40a-c41c594e8fa8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:36.675704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ef79d-5ff2-4b16-b829-c98c76129f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:36.676335 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ef79d-5ff2-4b16-b829-c98c76129f44 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:36.676617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ef79d-5ff2-4b16-b829-c98c76129f44 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:36.676942 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ef79d-5ff2-4b16-b829-c98c76129f44 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:36.677537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ef79d-5ff2-4b16-b829-c98c76129f44 None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:36.677839 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ef79d-5ff2-4b16-b829-c98c76129f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:36.678039 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e22738-7bba-4dd4-b956-9e57f900e056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:36.678694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e22738-7bba-4dd4-b956-9e57f900e056 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:36.678971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e22738-7bba-4dd4-b956-9e57f900e056 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:36.679231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ef79d-5ff2-4b16-b829-c98c76129f44 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:36.679327 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e22738-7bba-4dd4-b956-9e57f900e056 None admin] PATH_INFO: `/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/bfc14d32f4314c6e93d20a9c5a45c220/roles/15fbc4d519f64f4cacecc3b20fe826ae` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:36.692202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 317/649] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:36.692202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ef79d-5ff2-4b16-b829-c98c76129f44 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:36.701152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e22738-7bba-4dd4-b956-9e57f900e056 None admin] RBAC: Authorizing `identity:create_grant(project_id=7dee98edf19d478ca4f746ff85579fba, user_id=bfc14d32f4314c6e93d20a9c5a45c220, role_id=15fbc4d519f64f4cacecc3b20fe826ae)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:36.702949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e22738-7bba-4dd4-b956-9e57f900e056 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:36.712990 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:36.718368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 333/650] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:39:36 2022] PUT /identity/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/bfc14d32f4314c6e93d20a9c5a45c220/roles/15fbc4d519f64f4cacecc3b20fe826ae => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:39:36.726097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e22738-7bba-4dd4-b956-9e57f900e056 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:36.806953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:36.832353 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ef79d-5ff2-4b16-b829-c98c76129f44 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:36.863707 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e22738-7bba-4dd4-b956-9e57f900e056 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:36.885471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f300f-b6b3-4d6a-a020-9de8716ea52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:36.886382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f300f-b6b3-4d6a-a020-9de8716ea52d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:36.886675 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f300f-b6b3-4d6a-a020-9de8716ea52d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:36.886952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f300f-b6b3-4d6a-a020-9de8716ea52d None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:36.888253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f300f-b6b3-4d6a-a020-9de8716ea52d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:36.888565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f300f-b6b3-4d6a-a020-9de8716ea52d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:36.890037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f300f-b6b3-4d6a-a020-9de8716ea52d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:36.908923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc10969-ec91-4747-9e15-1e1c7905bff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:36.909645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc10969-ec91-4747-9e15-1e1c7905bff6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:36.909928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc10969-ec91-4747-9e15-1e1c7905bff6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:36.910203 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc10969-ec91-4747-9e15-1e1c7905bff6 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:36.911478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc10969-ec91-4747-9e15-1e1c7905bff6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:36.911787 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc10969-ec91-4747-9e15-1e1c7905bff6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:36.913869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc10969-ec91-4747-9e15-1e1c7905bff6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:36.914562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 318/651] 10.222.0.60 () {56 vars in 1137 bytes} [Mon Sep 19 14:39:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:36.915855 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f300f-b6b3-4d6a-a020-9de8716ea52d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:36.927701 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 334/652] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:36.930835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc10969-ec91-4747-9e15-1e1c7905bff6 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:37.011908 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:37.051381 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f300f-b6b3-4d6a-a020-9de8716ea52d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:37.083164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc10969-ec91-4747-9e15-1e1c7905bff6 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:37.092152 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c174e52-18db-4beb-b677-2a13c4aa368f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:37.092907 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c174e52-18db-4beb-b677-2a13c4aa368f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:37.093291 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c174e52-18db-4beb-b677-2a13c4aa368f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:37.093670 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c174e52-18db-4beb-b677-2a13c4aa368f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:37.104765 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c174e52-18db-4beb-b677-2a13c4aa368f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:37.106337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c174e52-18db-4beb-b677-2a13c4aa368f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:37.109026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:37.121241 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173e9a0-6e5e-408f-8980-e6d303854fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:37.122064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173e9a0-6e5e-408f-8980-e6d303854fa4 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:37.123166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173e9a0-6e5e-408f-8980-e6d303854fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:37.123585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173e9a0-6e5e-408f-8980-e6d303854fa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-3c174e52-18db-4beb-b677-2a13c4aa368f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:37.125909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:37.127065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:37.128306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:37.128306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:37.128306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:37.129753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 319/653] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:37.131384 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c174e52-18db-4beb-b677-2a13c4aa368f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:37.224654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173e9a0-6e5e-408f-8980-e6d303854fa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:37.226918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173e9a0-6e5e-408f-8980-e6d303854fa4 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:37.237836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c174e52-18db-4beb-b677-2a13c4aa368f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:37.274440 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6a53b-195c-42bc-b1bb-eaac414eacd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:37.275254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6a53b-195c-42bc-b1bb-eaac414eacd2 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:37.275674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6a53b-195c-42bc-b1bb-eaac414eacd2 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:37.276035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6a53b-195c-42bc-b1bb-eaac414eacd2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:37.285068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6a53b-195c-42bc-b1bb-eaac414eacd2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:37.286670 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6a53b-195c-42bc-b1bb-eaac414eacd2 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-36e6a53b-195c-42bc-b1bb-eaac414eacd2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:37.299945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:37.301175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:37.302392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:37.302392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:37.302392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:37.302392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:37.302392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:37.302392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:37.302392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:37.302392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:37.302392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 14:39:37.302392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:37.303815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 320/654] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:37.304807 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6a53b-195c-42bc-b1bb-eaac414eacd2 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:37.308965 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:37.359175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 335/655] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:37.360221 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173e9a0-6e5e-408f-8980-e6d303854fa4 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:37.412211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:37.431373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6a53b-195c-42bc-b1bb-eaac414eacd2 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:37.467598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173e9a0-6e5e-408f-8980-e6d303854fa4 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:37.474235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b25838-0af7-45fc-9735-fd8ebdcd0ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:37.475038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b25838-0af7-45fc-9735-fd8ebdcd0ccc None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:37.475472 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b25838-0af7-45fc-9735-fd8ebdcd0ccc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:37.475929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b25838-0af7-45fc-9735-fd8ebdcd0ccc None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:37.477153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b25838-0af7-45fc-9735-fd8ebdcd0ccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:37.477650 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b25838-0af7-45fc-9735-fd8ebdcd0ccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:37.479664 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b25838-0af7-45fc-9735-fd8ebdcd0ccc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:37.497280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 321/656] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:37.498358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b25838-0af7-45fc-9735-fd8ebdcd0ccc None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:37.510179 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d9ba7-eda3-4db0-9c5a-062925f9671a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:37.510922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d9ba7-eda3-4db0-9c5a-062925f9671a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:37.511306 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d9ba7-eda3-4db0-9c5a-062925f9671a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:37.511687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d9ba7-eda3-4db0-9c5a-062925f9671a None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:37.512694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d9ba7-eda3-4db0-9c5a-062925f9671a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:37.513514 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d9ba7-eda3-4db0-9c5a-062925f9671a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:37.515059 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d9ba7-eda3-4db0-9c5a-062925f9671a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:37.533802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 336/657] 10.222.0.60 () {56 vars in 1137 bytes} [Mon Sep 19 14:39:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:37.534933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d9ba7-eda3-4db0-9c5a-062925f9671a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:37.557083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:37.631567 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b25838-0af7-45fc-9735-fd8ebdcd0ccc None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:37.650287 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d9ba7-eda3-4db0-9c5a-062925f9671a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:37.654676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:37.682244 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9097593-c9e3-4efe-9a8d-be76c9461290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:37.682888 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9097593-c9e3-4efe-9a8d-be76c9461290 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:37.683172 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9097593-c9e3-4efe-9a8d-be76c9461290 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:37.683444 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9097593-c9e3-4efe-9a8d-be76c9461290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:37.695071 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3da46-aba1-42f5-9fdf-54226709c4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:37.695274 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9097593-c9e3-4efe-9a8d-be76c9461290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:37.696178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3da46-aba1-42f5-9fdf-54226709c4ee None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:37.696681 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9097593-c9e3-4efe-9a8d-be76c9461290 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:37.696793 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3da46-aba1-42f5-9fdf-54226709c4ee None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:37.697290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3da46-aba1-42f5-9fdf-54226709c4ee None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:37.707273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3da46-aba1-42f5-9fdf-54226709c4ee None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:37.709399 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3da46-aba1-42f5-9fdf-54226709c4ee None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c9097593-c9e3-4efe-9a8d-be76c9461290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:37.713231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:37.714267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:37.715184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:37.715184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:37.715184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:37.715184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 322/658] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:37.715872 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9097593-c9e3-4efe-9a8d-be76c9461290 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-f6f3da46-aba1-42f5-9fdf-54226709c4ee None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:37.727953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:37.729006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:37.730000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:37.730000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:37.730000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:37.730000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:37.730000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:37.730000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:37.730000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:37.730000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:37.730000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 19 14:39:37.730000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:37.733573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 337/659] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:37.734675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3da46-aba1-42f5-9fdf-54226709c4ee None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:37.838300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9097593-c9e3-4efe-9a8d-be76c9461290 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:37.840697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:37.868583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3da46-aba1-42f5-9fdf-54226709c4ee None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:37.878215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8f06f-a7c0-4620-ab07-6ae9b457528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:37.879112 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8f06f-a7c0-4620-ab07-6ae9b457528f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:37.879611 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8f06f-a7c0-4620-ab07-6ae9b457528f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:37.880098 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8f06f-a7c0-4620-ab07-6ae9b457528f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:37.912267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196f294-e8df-4ac6-9a18-8d5384138de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:37.912926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196f294-e8df-4ac6-9a18-8d5384138de7 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:37.913311 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196f294-e8df-4ac6-9a18-8d5384138de7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:37.913654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a196f294-e8df-4ac6-9a18-8d5384138de7 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:37.914524 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196f294-e8df-4ac6-9a18-8d5384138de7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:37.914914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196f294-e8df-4ac6-9a18-8d5384138de7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:37.916968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a196f294-e8df-4ac6-9a18-8d5384138de7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:37.922188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:37.932964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 338/660] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:37.933984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196f294-e8df-4ac6-9a18-8d5384138de7 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:37.971427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf8f06f-a7c0-4620-ab07-6ae9b457528f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:37.973308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf8f06f-a7c0-4620-ab07-6ae9b457528f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:38.066069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a196f294-e8df-4ac6-9a18-8d5384138de7 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:38.096983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 323/661] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:38.097631 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8f06f-a7c0-4620-ab07-6ae9b457528f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:38.109651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcd9b1-eb35-484d-959f-2d9588781009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:38.110718 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcd9b1-eb35-484d-959f-2d9588781009 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:38.111202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcd9b1-eb35-484d-959f-2d9588781009 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:38.111734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defcd9b1-eb35-484d-959f-2d9588781009 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:38.112677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defcd9b1-eb35-484d-959f-2d9588781009 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:38.113195 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defcd9b1-eb35-484d-959f-2d9588781009 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:38.115538 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defcd9b1-eb35-484d-959f-2d9588781009 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:38.126317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:38.140665 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 339/662] 10.222.0.60 () {56 vars in 1139 bytes} [Mon Sep 19 14:39:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:38.141896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcd9b1-eb35-484d-959f-2d9588781009 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:38.174338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:38.214769 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8f06f-a7c0-4620-ab07-6ae9b457528f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:38.252567 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1b4f5-7846-42e9-993c-d442d5a10f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:38.253212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1b4f5-7846-42e9-993c-d442d5a10f76 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:38.253649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1b4f5-7846-42e9-993c-d442d5a10f76 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:38.254008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1b4f5-7846-42e9-993c-d442d5a10f76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:38.263914 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1b4f5-7846-42e9-993c-d442d5a10f76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:38.265592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1b4f5-7846-42e9-993c-d442d5a10f76 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:38.275111 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defcd9b1-eb35-484d-959f-2d9588781009 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:38.278471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-a4e1b4f5-7846-42e9-993c-d442d5a10f76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:38.278471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:38.278471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:38.278471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:38.278471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:38.278471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:38.278471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:38.278471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:38.278471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:38.278471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:38.278471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:38.278471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:38.278471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:38.278471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:38.278471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:38.279369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:38.280451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:38.280451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:38.280451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:38.280451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:38.280451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:38.280451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:38.280451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:38.280451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:38.280451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 324/663] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:38.281193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1b4f5-7846-42e9-993c-d442d5a10f76 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:38.317776 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9f080-0600-464b-b250-a0ed0bae2506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:38.318405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9f080-0600-464b-b250-a0ed0bae2506 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:38.318684 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9f080-0600-464b-b250-a0ed0bae2506 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:38.319646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9f080-0600-464b-b250-a0ed0bae2506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:38.329936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9f080-0600-464b-b250-a0ed0bae2506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:38.332004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9f080-0600-464b-b250-a0ed0bae2506 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-0ed9f080-0600-464b-b250-a0ed0bae2506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:38.348105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:38.349126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:38.350037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:38.350037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:38.350037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:38.351968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 340/664] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:38.353238 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0305c-11f1-4f0d-98ae-248bb6643a93 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:38.353481 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0305c-11f1-4f0d-98ae-248bb6643a93 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:38.353758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0305c-11f1-4f0d-98ae-248bb6643a93 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:38.355008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 341/665] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:38.355695 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0305c-11f1-4f0d-98ae-248bb6643a93 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:38.392628 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1b4f5-7846-42e9-993c-d442d5a10f76 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:38.410612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:38.422016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:38.432904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801d37a-c531-4fdf-8dcc-292b005b93a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:38.433540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801d37a-c531-4fdf-8dcc-292b005b93a8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:38.433820 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801d37a-c531-4fdf-8dcc-292b005b93a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:38.434204 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801d37a-c531-4fdf-8dcc-292b005b93a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:38.465682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0305c-11f1-4f0d-98ae-248bb6643a93 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:38.506113 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe409e8b-1c1e-426d-9e2f-fbc5021e33b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:38.506506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801d37a-c531-4fdf-8dcc-292b005b93a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:38.507240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe409e8b-1c1e-426d-9e2f-fbc5021e33b1 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:38.507700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe409e8b-1c1e-426d-9e2f-fbc5021e33b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:38.508061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801d37a-c531-4fdf-8dcc-292b005b93a8 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:38.508390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe409e8b-1c1e-426d-9e2f-fbc5021e33b1 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:38.509253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe409e8b-1c1e-426d-9e2f-fbc5021e33b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:38.509749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe409e8b-1c1e-426d-9e2f-fbc5021e33b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:38.511160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe409e8b-1c1e-426d-9e2f-fbc5021e33b1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:38.524374 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 342/666] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:38.525194 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe409e8b-1c1e-426d-9e2f-fbc5021e33b1 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:38.617662 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 325/667] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:38.619251 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95820b13-1a3b-4989-9fd4-13e8cb7b2de2 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:38.619716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95820b13-1a3b-4989-9fd4-13e8cb7b2de2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:38.620192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95820b13-1a3b-4989-9fd4-13e8cb7b2de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:38.634275 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe409e8b-1c1e-426d-9e2f-fbc5021e33b1 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:38.638950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-95820b13-1a3b-4989-9fd4-13e8cb7b2de2 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:38.640353 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-95820b13-1a3b-4989-9fd4-13e8cb7b2de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:38.643958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:38.663385 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:38.672351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b948c4-53a4-4176-829e-fb5c330c79a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:38.672973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b948c4-53a4-4176-829e-fb5c330c79a3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:38.673251 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b948c4-53a4-4176-829e-fb5c330c79a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:38.673525 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b948c4-53a4-4176-829e-fb5c330c79a3 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:38.674424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b948c4-53a4-4176-829e-fb5c330c79a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:38.674725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b948c4-53a4-4176-829e-fb5c330c79a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:38.676121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b948c4-53a4-4176-829e-fb5c330c79a3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:38.688641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 343/668] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:38.689404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b948c4-53a4-4176-829e-fb5c330c79a3 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:38.718464 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 326/669] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:38.719305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95820b13-1a3b-4989-9fd4-13e8cb7b2de2 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:38.811122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b948c4-53a4-4176-829e-fb5c330c79a3 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:38.851328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c38867-9c4c-4600-a991-59d06bbe71e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:38.852456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c38867-9c4c-4600-a991-59d06bbe71e8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:38.852926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c38867-9c4c-4600-a991-59d06bbe71e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:38.853367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c38867-9c4c-4600-a991-59d06bbe71e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:38.853989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95820b13-1a3b-4989-9fd4-13e8cb7b2de2 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:38.854419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c38867-9c4c-4600-a991-59d06bbe71e8 None admin] RBAC: Adding query filter params (role.id=15fbc4d519f64f4cacecc3b20fe826ae, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=caab7aaa72634b358d717768fbcb48d8) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:38.854877 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c38867-9c4c-4600-a991-59d06bbe71e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:38.856596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c38867-9c4c-4600-a991-59d06bbe71e8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:38.863508 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 344/670] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:39:38 2022] GET /identity/v3/role_assignments?user.id=caab7aaa72634b358d717768fbcb48d8&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=15fbc4d519f64f4cacecc3b20fe826ae => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:38.864674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c38867-9c4c-4600-a991-59d06bbe71e8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:38.933707 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:38.935545 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62181d-8a71-4191-893c-ca5425f6b26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:38.937843 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62181d-8a71-4191-893c-ca5425f6b26d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:38.938807 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62181d-8a71-4191-893c-ca5425f6b26d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:38.941768 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62181d-8a71-4191-893c-ca5425f6b26d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:38.965016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62181d-8a71-4191-893c-ca5425f6b26d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:38.968474 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62181d-8a71-4191-893c-ca5425f6b26d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-3b62181d-8a71-4191-893c-ca5425f6b26d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:38.996891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:38.998035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:38.998991 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:38.998991 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:38.998991 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:39.004850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 327/671] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:39.005997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62181d-8a71-4191-893c-ca5425f6b26d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:39.022750 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:39.060656 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c38867-9c4c-4600-a991-59d06bbe71e8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:39.101825 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c981a37-74b5-4ff8-86a7-dbd540bab391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:39.102467 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981a37-74b5-4ff8-86a7-dbd540bab391 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:39.102749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981a37-74b5-4ff8-86a7-dbd540bab391 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:39.103025 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981a37-74b5-4ff8-86a7-dbd540bab391 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:39.110804 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c981a37-74b5-4ff8-86a7-dbd540bab391 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:39.112312 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c981a37-74b5-4ff8-86a7-dbd540bab391 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:39.125213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-5c981a37-74b5-4ff8-86a7-dbd540bab391 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:39.125213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:39.125213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:39.125213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:39.125213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:39.125213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:39.125213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:39.125213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:39.125213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:39.125213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:39.125213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:39.125213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:39.125213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:39.125213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:39.125213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:39.126321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:39.127408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:39.127408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:39.127408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:39.127408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:39.127408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:39.127408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:39.127408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:39.127408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:39.127408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 345/672] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:39.127408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c981a37-74b5-4ff8-86a7-dbd540bab391 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:39.145092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62181d-8a71-4191-893c-ca5425f6b26d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:39.188222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ef647-b4c8-4c7d-963f-9eb59575419f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:39.188895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ef647-b4c8-4c7d-963f-9eb59575419f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:39.189126 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ef647-b4c8-4c7d-963f-9eb59575419f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:39.189402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ef647-b4c8-4c7d-963f-9eb59575419f None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:39.190049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ef647-b4c8-4c7d-963f-9eb59575419f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:39.190355 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ef647-b4c8-4c7d-963f-9eb59575419f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:39.191743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ef647-b4c8-4c7d-963f-9eb59575419f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:39.205902 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 328/673] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:39.207538 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2600c3-4c52-4da5-a19a-bbdd0c04707b None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:39.207817 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2600c3-4c52-4da5-a19a-bbdd0c04707b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:39.208172 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2600c3-4c52-4da5-a19a-bbdd0c04707b None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:39.209522 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 329/674] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:39.209986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2600c3-4c52-4da5-a19a-bbdd0c04707b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:39.251013 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:39.260507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c981a37-74b5-4ff8-86a7-dbd540bab391 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:39.284082 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:39.301367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e2ac6-7e7f-4138-a946-9b5c70d27da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:39.301994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e2ac6-7e7f-4138-a946-9b5c70d27da8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:39.302271 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e2ac6-7e7f-4138-a946-9b5c70d27da8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:39.302535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037e2ac6-7e7f-4138-a946-9b5c70d27da8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:39.314038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037e2ac6-7e7f-4138-a946-9b5c70d27da8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-037e2ac6-7e7f-4138-a946-9b5c70d27da8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:39.319264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:39.320299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:39.321206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:39.321206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:39.321206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:39.321206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:39.321206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:39.321206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:39.321206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:39.321206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:39.321206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:39.321206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:39.321206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:39.321206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:39.321206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 346/675] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:39:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:39.321206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e2ac6-7e7f-4138-a946-9b5c70d27da8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:39.328446 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2600c3-4c52-4da5-a19a-bbdd0c04707b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:39.366321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c25e82-80e1-4474-86e3-d6ca56df3989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:39.366938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c25e82-80e1-4474-86e3-d6ca56df3989 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:39.367361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c25e82-80e1-4474-86e3-d6ca56df3989 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:39.367798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c25e82-80e1-4474-86e3-d6ca56df3989 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:39.368389 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c25e82-80e1-4474-86e3-d6ca56df3989 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:39.368708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c25e82-80e1-4474-86e3-d6ca56df3989 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:39.370154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c25e82-80e1-4474-86e3-d6ca56df3989 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:39.387253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 330/676] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:39.387253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb02b91-d84c-4493-8351-71f6444a7e93 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:39.387253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb02b91-d84c-4493-8351-71f6444a7e93 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:39.387253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb02b91-d84c-4493-8351-71f6444a7e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:39.402865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6cb02b91-d84c-4493-8351-71f6444a7e93 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:39.404730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6cb02b91-d84c-4493-8351-71f6444a7e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:39.433871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037e2ac6-7e7f-4138-a946-9b5c70d27da8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:39.471784 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991771a-8c5f-47ca-9343-7f7b664131fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:39.472362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991771a-8c5f-47ca-9343-7f7b664131fc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:39.472646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991771a-8c5f-47ca-9343-7f7b664131fc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:39.472921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991771a-8c5f-47ca-9343-7f7b664131fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:39.476518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 331/677] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:39.481011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb02b91-d84c-4493-8351-71f6444a7e93 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:39.482131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991771a-8c5f-47ca-9343-7f7b664131fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:39.483857 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991771a-8c5f-47ca-9343-7f7b664131fc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:39.498615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-7991771a-8c5f-47ca-9343-7f7b664131fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:39.498615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:39.498615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:39.498615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:39.498615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:39.498615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:39.498615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:39.498615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:39.498615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:39.498615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:39.498615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:39.498615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:39.498615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:39.498615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:39.498615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:39.499577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:39.500541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:39.500541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:39.500541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:39.500541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:39.500541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:39.500541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:39.500541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:39.500541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:39.507969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 347/678] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:39.508846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991771a-8c5f-47ca-9343-7f7b664131fc None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:39.527845 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:39.559500 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:39.618375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb02b91-d84c-4493-8351-71f6444a7e93 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:39.631224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991771a-8c5f-47ca-9343-7f7b664131fc None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:39.666265 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869189a5-8431-4bb5-89df-78ecbe459078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:39.666881 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869189a5-8431-4bb5-89df-78ecbe459078 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:39.667577 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869189a5-8431-4bb5-89df-78ecbe459078 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:39.667879 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869189a5-8431-4bb5-89df-78ecbe459078 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:39.668601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869189a5-8431-4bb5-89df-78ecbe459078 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:39.671948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869189a5-8431-4bb5-89df-78ecbe459078 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:39.673665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869189a5-8431-4bb5-89df-78ecbe459078 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:39.681978 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a5ef5-2a5d-4c97-8066-b4e86460a7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:39.682997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a5ef5-2a5d-4c97-8066-b4e86460a7b5 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:39.682997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a5ef5-2a5d-4c97-8066-b4e86460a7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:39.683260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a5ef5-2a5d-4c97-8066-b4e86460a7b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:39.684051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a5ef5-2a5d-4c97-8066-b4e86460a7b5 None admin] RBAC: Adding query filter params (role.id=15fbc4d519f64f4cacecc3b20fe826ae, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=006fe9ba3b0a40f29fa2be7620e548c4) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:39.684363 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a5ef5-2a5d-4c97-8066-b4e86460a7b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:39.685945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a5ef5-2a5d-4c97-8066-b4e86460a7b5 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:39.690313 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 332/679] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:39.691495 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869189a5-8431-4bb5-89df-78ecbe459078 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:39.693130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 348/680] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:39:39 2022] GET /identity/v3/role_assignments?user.id=006fe9ba3b0a40f29fa2be7620e548c4&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=15fbc4d519f64f4cacecc3b20fe826ae => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:39.699507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a5ef5-2a5d-4c97-8066-b4e86460a7b5 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:39.811455 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:39.827899 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869189a5-8431-4bb5-89df-78ecbe459078 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:39.848974 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a5ef5-2a5d-4c97-8066-b4e86460a7b5 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:39.852106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:39.870382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f93459-db42-4a0a-99ef-fba1f84713cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:39.871255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f93459-db42-4a0a-99ef-fba1f84713cd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:39.871679 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f93459-db42-4a0a-99ef-fba1f84713cd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:39.872121 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f93459-db42-4a0a-99ef-fba1f84713cd None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:39.873112 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f93459-db42-4a0a-99ef-fba1f84713cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:39.873598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f93459-db42-4a0a-99ef-fba1f84713cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:39.875216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f93459-db42-4a0a-99ef-fba1f84713cd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:39.891886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 333/681] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:39.893152 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f93459-db42-4a0a-99ef-fba1f84713cd None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:39.902591 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f9680-9765-4472-92ee-1e73d1e1ed93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:39.903242 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f9680-9765-4472-92ee-1e73d1e1ed93 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:39.904095 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f9680-9765-4472-92ee-1e73d1e1ed93 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:39.904374 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942f9680-9765-4472-92ee-1e73d1e1ed93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:39.921334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f9680-9765-4472-92ee-1e73d1e1ed93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:39.924372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942f9680-9765-4472-92ee-1e73d1e1ed93 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-942f9680-9765-4472-92ee-1e73d1e1ed93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:39.945346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:39.946757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:39.947660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:39.947660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:39.947660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:39.949850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 349/682] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:39.951534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f9680-9765-4472-92ee-1e73d1e1ed93 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:40.017815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f93459-db42-4a0a-99ef-fba1f84713cd None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:40.056966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dcc2f-491f-4252-8578-fc14387dd407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:40.057665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dcc2f-491f-4252-8578-fc14387dd407 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:40.057950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dcc2f-491f-4252-8578-fc14387dd407 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:40.058231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dcc2f-491f-4252-8578-fc14387dd407 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:40.070122 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dcc2f-491f-4252-8578-fc14387dd407 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c64dcc2f-491f-4252-8578-fc14387dd407 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:40.074935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:40.076733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:40.077580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:40.077580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:40.077580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:40.077580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:40.077580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:40.077580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:40.077580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:40.077580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:40.077580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:40.077580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:40.077580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:40.077580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:40.077580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 334/683] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:40.077580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dcc2f-491f-4252-8578-fc14387dd407 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:40.101190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942f9680-9765-4472-92ee-1e73d1e1ed93 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:40.128260 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:40.142450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:40.154634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd26e11-a006-45c4-b235-323dc3e6c91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:40.155599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd26e11-a006-45c4-b235-323dc3e6c91d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:40.156137 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd26e11-a006-45c4-b235-323dc3e6c91d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:40.156841 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd26e11-a006-45c4-b235-323dc3e6c91d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:40.158376 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd26e11-a006-45c4-b235-323dc3e6c91d None admin] RBAC: Adding query filter params (role.id=15fbc4d519f64f4cacecc3b20fe826ae, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=2fbef7b24f134f5eb1193aed1b7ee69c) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:40.158892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd26e11-a006-45c4-b235-323dc3e6c91d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:40.161032 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd26e11-a006-45c4-b235-323dc3e6c91d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:40.170008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 350/684] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:39:39 2022] GET /identity/v3/role_assignments?user.id=2fbef7b24f134f5eb1193aed1b7ee69c&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=15fbc4d519f64f4cacecc3b20fe826ae => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:40.171531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd26e11-a006-45c4-b235-323dc3e6c91d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:40.234682 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dcc2f-491f-4252-8578-fc14387dd407 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:40.318300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26662d9-2849-40a2-af86-0253d06813f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:40.318944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26662d9-2849-40a2-af86-0253d06813f5 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:40.319231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26662d9-2849-40a2-af86-0253d06813f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:40.319510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26662d9-2849-40a2-af86-0253d06813f5 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:40.320255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26662d9-2849-40a2-af86-0253d06813f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:40.320504 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26662d9-2849-40a2-af86-0253d06813f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:40.322096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26662d9-2849-40a2-af86-0253d06813f5 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:40.332654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd26e11-a006-45c4-b235-323dc3e6c91d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:40.350547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 335/685] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:40.358332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26662d9-2849-40a2-af86-0253d06813f5 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:40.388547 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f652a-2f75-4f0e-a6cf-b9e8d1f134ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:40.390037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f652a-2f75-4f0e-a6cf-b9e8d1f134ef None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:40.390037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f652a-2f75-4f0e-a6cf-b9e8d1f134ef None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:40.390037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132f652a-2f75-4f0e-a6cf-b9e8d1f134ef None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:40.390454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f652a-2f75-4f0e-a6cf-b9e8d1f134ef None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:40.390818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f652a-2f75-4f0e-a6cf-b9e8d1f134ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:40.393250 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132f652a-2f75-4f0e-a6cf-b9e8d1f134ef None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:40.404411 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 351/686] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:40.409987 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f652a-2f75-4f0e-a6cf-b9e8d1f134ef None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:40.470874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:40.499127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26662d9-2849-40a2-af86-0253d06813f5 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:40.512228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:40.528248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132f652a-2f75-4f0e-a6cf-b9e8d1f134ef None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:40.544261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877220c-1363-409f-afa5-bbbccbdc58fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:40.544908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877220c-1363-409f-afa5-bbbccbdc58fd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:40.545972 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877220c-1363-409f-afa5-bbbccbdc58fd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:40.546247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877220c-1363-409f-afa5-bbbccbdc58fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:40.559979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877220c-1363-409f-afa5-bbbccbdc58fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:40.562224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a877220c-1363-409f-afa5-bbbccbdc58fd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:40.572975 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d428f2-5aaa-4404-8da1-c6736fa1edfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:40.573540 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d428f2-5aaa-4404-8da1-c6736fa1edfe None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:40.573820 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d428f2-5aaa-4404-8da1-c6736fa1edfe None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:40.574094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d428f2-5aaa-4404-8da1-c6736fa1edfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-a877220c-1363-409f-afa5-bbbccbdc58fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:40.576636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:40.577578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:40.578550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:40.578550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:40.578550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:40.578550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:40.578550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:40.578550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:40.578550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:40.578550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:40.578550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:39:40.578550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:40.579098 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 336/687] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Sep 19 14:39:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:39:40.583523 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877220c-1363-409f-afa5-bbbccbdc58fd None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:40.588328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d428f2-5aaa-4404-8da1-c6736fa1edfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:40.589890 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d428f2-5aaa-4404-8da1-c6736fa1edfe None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-23d428f2-5aaa-4404-8da1-c6736fa1edfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:40.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:40.610346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:40.612719 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:40.612719 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:40.612719 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:40.615859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 352/688] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:39:40.619773 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d428f2-5aaa-4404-8da1-c6736fa1edfe None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:40.708330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877220c-1363-409f-afa5-bbbccbdc58fd None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:40.740821 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d428f2-5aaa-4404-8da1-c6736fa1edfe None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:40.749148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:40.751340 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd5328-4ddc-40f7-a902-fe4bcee69fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:40.752001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd5328-4ddc-40f7-a902-fe4bcee69fbc None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:40.752285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd5328-4ddc-40f7-a902-fe4bcee69fbc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:40.752557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd5328-4ddc-40f7-a902-fe4bcee69fbc None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:40.754122 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd5328-4ddc-40f7-a902-fe4bcee69fbc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:40.754428 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd5328-4ddc-40f7-a902-fe4bcee69fbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:40.756171 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd5328-4ddc-40f7-a902-fe4bcee69fbc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:40.779094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf746e-0ec6-4047-91b8-fb094c88856b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:40.779718 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf746e-0ec6-4047-91b8-fb094c88856b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:40.780562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf746e-0ec6-4047-91b8-fb094c88856b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:40.780848 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf746e-0ec6-4047-91b8-fb094c88856b None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:40.781491 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf746e-0ec6-4047-91b8-fb094c88856b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:40.781795 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf746e-0ec6-4047-91b8-fb094c88856b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:40.783164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bf746e-0ec6-4047-91b8-fb094c88856b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:40.787394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 337/689] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Sep 19 14:39:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:39:40.793148 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd5328-4ddc-40f7-a902-fe4bcee69fbc None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:40.797122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 353/690] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:39:40.815546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf746e-0ec6-4047-91b8-fb094c88856b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:40.818602 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:40.955311 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf746e-0ec6-4047-91b8-fb094c88856b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:40.995997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce82546-bf4c-4c86-9e8a-936649bfa6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:40.996637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce82546-bf4c-4c86-9e8a-936649bfa6df None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:40.996915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce82546-bf4c-4c86-9e8a-936649bfa6df None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:40.997215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce82546-bf4c-4c86-9e8a-936649bfa6df None admin] PATH_INFO: `/v3/users/swift` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:41.014662 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce82546-bf4c-4c86-9e8a-936649bfa6df None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:41.016151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce82546-bf4c-4c86-9e8a-936649bfa6df None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:41.025152 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd5328-4ddc-40f7-a902-fe4bcee69fbc None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-4ce82546-bf4c-4c86-9e8a-936649bfa6df None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:41.048170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:41.049453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:41.050502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:41.050502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:41.050502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:41.050502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:41.050502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:41.050502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:41.050502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:41.050502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:41.050502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 14:39:41.050502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:41.054427 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 354/691] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:39:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:39:41.055681 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce82546-bf4c-4c86-9e8a-936649bfa6df None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:41.089610 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:41.114469 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a2748-419b-4799-a488-8372c0d10fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:41.115093 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a2748-419b-4799-a488-8372c0d10fa4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:41.115375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a2748-419b-4799-a488-8372c0d10fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:41.115648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a2748-419b-4799-a488-8372c0d10fa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:41.167576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce82546-bf4c-4c86-9e8a-936649bfa6df None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:41.205689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf3c37-59c2-466f-9b73-d3a65146b4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:41.206223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf3c37-59c2-466f-9b73-d3a65146b4ca None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:41.206503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf3c37-59c2-466f-9b73-d3a65146b4ca None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:41.206774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf3c37-59c2-466f-9b73-d3a65146b4ca None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:41.207496 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf3c37-59c2-466f-9b73-d3a65146b4ca None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:41.207803 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf3c37-59c2-466f-9b73-d3a65146b4ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:41.209256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf3c37-59c2-466f-9b73-d3a65146b4ca None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:41.234124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 355/692] 10.222.0.60 () {56 vars in 1135 bytes} [Mon Sep 19 14:39:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:41.235696 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf3c37-59c2-466f-9b73-d3a65146b4ca None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:41.271433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a2748-419b-4799-a488-8372c0d10fa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:41.273358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a2748-419b-4799-a488-8372c0d10fa4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:41.316405 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:41.333244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:41.351517 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf3c37-59c2-466f-9b73-d3a65146b4ca None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:41.394114 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4561d-7820-46c6-be44-7b2dae185e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:41.395172 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4561d-7820-46c6-be44-7b2dae185e16 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:41.395528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4561d-7820-46c6-be44-7b2dae185e16 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:41.395912 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4561d-7820-46c6-be44-7b2dae185e16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:41.455884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 338/693] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 19 14:39:41.458275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65ce58-92f5-4411-9299-da2a986e50a1 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:41.458745 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65ce58-92f5-4411-9299-da2a986e50a1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:41.459437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65ce58-92f5-4411-9299-da2a986e50a1 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:41.466133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 339/694] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:41.466133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65ce58-92f5-4411-9299-da2a986e50a1 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:41.483812 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc4561d-7820-46c6-be44-7b2dae185e16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:41.485457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc4561d-7820-46c6-be44-7b2dae185e16 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:41.584186 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:41.603608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65ce58-92f5-4411-9299-da2a986e50a1 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:41.609119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 356/695] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 19 14:39:41.610747 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00f8d3-0993-47ef-b034-4741d67d146c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:41.611267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00f8d3-0993-47ef-b034-4741d67d146c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:41.611741 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00f8d3-0993-47ef-b034-4741d67d146c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:41.625399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:41.638726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5a00f8d3-0993-47ef-b034-4741d67d146c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:41.640754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5a00f8d3-0993-47ef-b034-4741d67d146c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:41.647308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d8a69-9789-4b4f-9070-1a35615781a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:41.647995 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d8a69-9789-4b4f-9070-1a35615781a4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:41.648278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d8a69-9789-4b4f-9070-1a35615781a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:41.648609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d8a69-9789-4b4f-9070-1a35615781a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:41.662721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d8a69-9789-4b4f-9070-1a35615781a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:41.664599 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169d8a69-9789-4b4f-9070-1a35615781a4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-169d8a69-9789-4b4f-9070-1a35615781a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:41.684518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:41.685549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:41.686510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:41.686510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:41.686510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:41.687098 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 340/696] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:41.687707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d8a69-9789-4b4f-9070-1a35615781a4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:41.732460 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 357/697] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:41.732460 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00f8d3-0993-47ef-b034-4741d67d146c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:41.812120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d8a69-9789-4b4f-9070-1a35615781a4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:41.830919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00f8d3-0993-47ef-b034-4741d67d146c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:41.846263 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:41.862561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02217584-e3a4-43e3-a6fe-e59e860a1596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:41.863571 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02217584-e3a4-43e3-a6fe-e59e860a1596 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:41.864105 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02217584-e3a4-43e3-a6fe-e59e860a1596 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:41.864584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02217584-e3a4-43e3-a6fe-e59e860a1596 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:41.867752 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e3c40-8201-41b1-b4e4-b26d4c63add4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:41.868388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e3c40-8201-41b1-b4e4-b26d4c63add4 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:41.868666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e3c40-8201-41b1-b4e4-b26d4c63add4 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:41.868993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e3c40-8201-41b1-b4e4-b26d4c63add4 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:41.869637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e3c40-8201-41b1-b4e4-b26d4c63add4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:41.869959 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e3c40-8201-41b1-b4e4-b26d4c63add4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:41.871311 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e3c40-8201-41b1-b4e4-b26d4c63add4 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:41.880493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02217584-e3a4-43e3-a6fe-e59e860a1596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:41.882485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02217584-e3a4-43e3-a6fe-e59e860a1596 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:41.883647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 358/698] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:41.889293 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e3c40-8201-41b1-b4e4-b26d4c63add4 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-02217584-e3a4-43e3-a6fe-e59e860a1596 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:41.900200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:41.901230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:41.902184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:41.902184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:41.902184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:41.904919 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 341/699] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:41.913633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02217584-e3a4-43e3-a6fe-e59e860a1596 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:41.965794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:42.004569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e3c40-8201-41b1-b4e4-b26d4c63add4 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:42.048756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87729e1-af5c-4a7a-a359-2a9d90ef67e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:42.049667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87729e1-af5c-4a7a-a359-2a9d90ef67e0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:42.049955 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87729e1-af5c-4a7a-a359-2a9d90ef67e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:42.050230 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87729e1-af5c-4a7a-a359-2a9d90ef67e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:42.062347 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87729e1-af5c-4a7a-a359-2a9d90ef67e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:42.063780 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87729e1-af5c-4a7a-a359-2a9d90ef67e0 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:42.082704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-a87729e1-af5c-4a7a-a359-2a9d90ef67e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:42.082704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:42.082704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:42.082704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:42.082704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:42.082704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:42.082704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:42.082704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:42.082704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:42.082704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:42.082704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:42.082704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:42.082704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:42.082704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:42.082704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:42.083648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:42.084576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:42.084576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:42.084576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:42.084576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:42.084576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:42.084576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:42.084576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:42.084576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:42.087941 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 359/700] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:42.096724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87729e1-af5c-4a7a-a359-2a9d90ef67e0 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:42.109222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02217584-e3a4-43e3-a6fe-e59e860a1596 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:42.136882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:42.206530 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc6324-8a93-4bb8-9897-490514fb997f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:42.207538 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc6324-8a93-4bb8-9897-490514fb997f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:42.208662 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc6324-8a93-4bb8-9897-490514fb997f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:42.208963 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc6324-8a93-4bb8-9897-490514fb997f None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:42.209636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc6324-8a93-4bb8-9897-490514fb997f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:42.209945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc6324-8a93-4bb8-9897-490514fb997f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:42.212597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc6324-8a93-4bb8-9897-490514fb997f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:42.221654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87729e1-af5c-4a7a-a359-2a9d90ef67e0 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:42.234396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 342/701] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:42.235786 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc6324-8a93-4bb8-9897-490514fb997f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:42.265353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e5344-0c03-4c44-8a4d-18fd1d8dbeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:42.266567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e5344-0c03-4c44-8a4d-18fd1d8dbeae None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:42.267590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e5344-0c03-4c44-8a4d-18fd1d8dbeae None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:42.268514 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e5344-0c03-4c44-8a4d-18fd1d8dbeae None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:42.269518 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e5344-0c03-4c44-8a4d-18fd1d8dbeae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:42.270141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e5344-0c03-4c44-8a4d-18fd1d8dbeae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:42.271851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e5344-0c03-4c44-8a4d-18fd1d8dbeae None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:42.295055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 360/702] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:42.295055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e5344-0c03-4c44-8a4d-18fd1d8dbeae None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:42.342825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:42.362993 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc6324-8a93-4bb8-9897-490514fb997f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:42.400479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e6339-b7fe-42a8-9247-e8cf386dd719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:42.401113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e6339-b7fe-42a8-9247-e8cf386dd719 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:42.401394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e6339-b7fe-42a8-9247-e8cf386dd719 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:42.401671 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e6339-b7fe-42a8-9247-e8cf386dd719 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:42.408396 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:42.413272 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e6339-b7fe-42a8-9247-e8cf386dd719 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:42.414743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e6339-b7fe-42a8-9247-e8cf386dd719 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:42.417667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e5344-0c03-4c44-8a4d-18fd1d8dbeae None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-e62e6339-b7fe-42a8-9247-e8cf386dd719 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:42.430661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:42.436741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:42.437904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:42.437904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:42.437904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:42.437904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 343/703] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:42.437904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e6339-b7fe-42a8-9247-e8cf386dd719 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:42.466800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f10e4-2dcf-4fae-8122-38ce70b02784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:42.467556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f10e4-2dcf-4fae-8122-38ce70b02784 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:42.467860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f10e4-2dcf-4fae-8122-38ce70b02784 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:42.468141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f10e4-2dcf-4fae-8122-38ce70b02784 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:42.475419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f10e4-2dcf-4fae-8122-38ce70b02784 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:42.476982 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f10e4-2dcf-4fae-8122-38ce70b02784 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:42.489808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-fc6f10e4-2dcf-4fae-8122-38ce70b02784 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:42.489808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:42.489808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:42.489808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:42.489808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:42.489808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:42.489808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:42.489808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:42.489808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:42.489808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:42.489808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:42.489808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:42.489808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:42.489808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:42.489808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:42.490896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:42.492146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:42.492146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:42.492146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:42.492146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:42.492146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:42.492146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:42.492146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:42.492146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:42.496117 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 361/704] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:42.497306 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11490617-5f5c-4bfe-8800-d8a036dbd325 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:42.497591 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11490617-5f5c-4bfe-8800-d8a036dbd325 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:42.497868 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11490617-5f5c-4bfe-8800-d8a036dbd325 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:42.499289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 362/705] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:42.499817 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11490617-5f5c-4bfe-8800-d8a036dbd325 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:42.551045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e6339-b7fe-42a8-9247-e8cf386dd719 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:42.586278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9ada4-f14e-4ffa-a9ca-4925b784b3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:42.586884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9ada4-f14e-4ffa-a9ca-4925b784b3b8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:42.587163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9ada4-f14e-4ffa-a9ca-4925b784b3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:42.587445 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9ada4-f14e-4ffa-a9ca-4925b784b3b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:42.588190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9ada4-f14e-4ffa-a9ca-4925b784b3b8 None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=59b5d2fd362b417c9b9be75f88a667ad) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:42.588498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9ada4-f14e-4ffa-a9ca-4925b784b3b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:42.589929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9ada4-f14e-4ffa-a9ca-4925b784b3b8 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:42.592354 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:42.597816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 344/706] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:39:42 2022] GET /identity/v3/role_assignments?user.id=59b5d2fd362b417c9b9be75f88a667ad&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:42.598658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9ada4-f14e-4ffa-a9ca-4925b784b3b8 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:42.623076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11490617-5f5c-4bfe-8800-d8a036dbd325 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:42.672053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711611a-e23b-423e-8e98-3c817b0fcae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:42.672789 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711611a-e23b-423e-8e98-3c817b0fcae1 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:42.673066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711611a-e23b-423e-8e98-3c817b0fcae1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:42.673465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711611a-e23b-423e-8e98-3c817b0fcae1 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:42.674741 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711611a-e23b-423e-8e98-3c817b0fcae1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:42.675082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711611a-e23b-423e-8e98-3c817b0fcae1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:42.676758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711611a-e23b-423e-8e98-3c817b0fcae1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:42.695588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 363/707] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:42.697073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca2926-9efd-4e4b-b89c-168bb4c506a4 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:42.697347 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca2926-9efd-4e4b-b89c-168bb4c506a4 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:42.697702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca2926-9efd-4e4b-b89c-168bb4c506a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:42.718603 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e1ca2926-9efd-4e4b-b89c-168bb4c506a4 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:42.720000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e1ca2926-9efd-4e4b-b89c-168bb4c506a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:42.731469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:42.748503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9ada4-f14e-4ffa-a9ca-4925b784b3b8 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:42.790832 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fcfb1-c681-4971-bc1d-40410e0c5700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:42.791871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fcfb1-c681-4971-bc1d-40410e0c5700 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:42.792255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fcfb1-c681-4971-bc1d-40410e0c5700 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:42.792625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70fcfb1-c681-4971-bc1d-40410e0c5700 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:42.793419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fcfb1-c681-4971-bc1d-40410e0c5700 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:42.793756 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fcfb1-c681-4971-bc1d-40410e0c5700 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:42.795198 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70fcfb1-c681-4971-bc1d-40410e0c5700 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:42.799980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 364/708] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:42.801289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca2926-9efd-4e4b-b89c-168bb4c506a4 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:42.809435 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 345/709] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:42.815205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fcfb1-c681-4971-bc1d-40410e0c5700 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:42.896305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:42.913979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca2926-9efd-4e4b-b89c-168bb4c506a4 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:42.935566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70fcfb1-c681-4971-bc1d-40410e0c5700 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:42.976433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59333b2-6d8c-42d0-8239-ed7bf45dcb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:42.977236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59333b2-6d8c-42d0-8239-ed7bf45dcb57 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:42.977614 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59333b2-6d8c-42d0-8239-ed7bf45dcb57 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:42.977976 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59333b2-6d8c-42d0-8239-ed7bf45dcb57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:42.978831 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59333b2-6d8c-42d0-8239-ed7bf45dcb57 None admin] RBAC: Adding query filter params (role.id=15fbc4d519f64f4cacecc3b20fe826ae, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=bfc14d32f4314c6e93d20a9c5a45c220) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:42.979304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59333b2-6d8c-42d0-8239-ed7bf45dcb57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:42.981059 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59333b2-6d8c-42d0-8239-ed7bf45dcb57 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:42.987054 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6135dc3-b037-4f40-8c59-e90ad85b9d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:42.987685 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6135dc3-b037-4f40-8c59-e90ad85b9d8b None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:42.987969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 346/710] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:39:42 2022] GET /identity/v3/role_assignments?user.id=bfc14d32f4314c6e93d20a9c5a45c220&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=15fbc4d519f64f4cacecc3b20fe826ae => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:42.988090 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6135dc3-b037-4f40-8c59-e90ad85b9d8b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:42.988339 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6135dc3-b037-4f40-8c59-e90ad85b9d8b None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:42.988990 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6135dc3-b037-4f40-8c59-e90ad85b9d8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:42.990423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6135dc3-b037-4f40-8c59-e90ad85b9d8b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:43.012021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:43.017955 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 365/711] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:39:42 2022] POST /identity/v3/users => generated 254 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:39:43.030285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a8047-c2b1-470c-a7be-331794988300 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:43.030815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a8047-c2b1-470c-a7be-331794988300 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:43.031267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a8047-c2b1-470c-a7be-331794988300 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:43.033220 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 347/712] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:43.039592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecdf90-5bd0-4cf5-aebd-a686f7b2fb51 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:43.039922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecdf90-5bd0-4cf5-aebd-a686f7b2fb51 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:43.040214 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aecdf90-5bd0-4cf5-aebd-a686f7b2fb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:43.068010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-6aecdf90-5bd0-4cf5-aebd-a686f7b2fb51 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:43.069599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6aecdf90-5bd0-4cf5-aebd-a686f7b2fb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:43.186288 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 366/713] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:43.219879 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a8047-c2b1-470c-a7be-331794988300 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:43.350659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a8047-c2b1-470c-a7be-331794988300 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:43.374439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:43.386592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06fbb6-f9d9-4911-b1f9-5682a1876184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:43.387234 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06fbb6-f9d9-4911-b1f9-5682a1876184 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:43.387514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06fbb6-f9d9-4911-b1f9-5682a1876184 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:43.388333 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06fbb6-f9d9-4911-b1f9-5682a1876184 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:43.389016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06fbb6-f9d9-4911-b1f9-5682a1876184 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:43.390415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06fbb6-f9d9-4911-b1f9-5682a1876184 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-7e06fbb6-f9d9-4911-b1f9-5682a1876184 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:43.402196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:43.403261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:43.404784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:43.404784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:43.404784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:43.404784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:43.404784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:43.404784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:43.404784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:43.404784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:43.404784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:43.404784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:43.404784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:39:43.404784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:39:43.404784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 14:39:43.404784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:43.404784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 348/714] 10.222.0.60 () {56 vars in 1129 bytes} [Mon Sep 19 14:39:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:43.409886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecdf90-5bd0-4cf5-aebd-a686f7b2fb51 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:43.498758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06fbb6-f9d9-4911-b1f9-5682a1876184 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:43.533681 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aecdf90-5bd0-4cf5-aebd-a686f7b2fb51 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:43.573302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:43.582320 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95c2e0-0e3a-444c-b822-9e491e8b9d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:43.582959 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95c2e0-0e3a-444c-b822-9e491e8b9d93 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:43.583241 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95c2e0-0e3a-444c-b822-9e491e8b9d93 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:43.583575 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95c2e0-0e3a-444c-b822-9e491e8b9d93 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:43.585134 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95c2e0-0e3a-444c-b822-9e491e8b9d93 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:43.585438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95c2e0-0e3a-444c-b822-9e491e8b9d93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:43.587511 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95c2e0-0e3a-444c-b822-9e491e8b9d93 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:43.594478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 367/715] 10.222.0.60 () {56 vars in 1124 bytes} [Mon Sep 19 14:39:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:43.605024 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95c2e0-0e3a-444c-b822-9e491e8b9d93 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:43.620352 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06fbb6-f9d9-4911-b1f9-5682a1876184 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:43.667429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826b34f-c641-40b6-b161-616d88fd8a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:43.668259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826b34f-c641-40b6-b161-616d88fd8a4e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:43.668547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826b34f-c641-40b6-b161-616d88fd8a4e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:43.668821 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1826b34f-c641-40b6-b161-616d88fd8a4e None admin] PATH_INFO: `/v3/services/network` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:43.669964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1826b34f-c641-40b6-b161-616d88fd8a4e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:43.671483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1826b34f-c641-40b6-b161-616d88fd8a4e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-1826b34f-c641-40b6-b161-616d88fd8a4e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:43.694206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:43.695302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:43.696228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:43.696228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:43.696228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:43.696228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:43.696228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:43.696228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:43.696228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:43.696228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:43.696228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:43.696228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:43.696228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:39:43.696228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:39:43.696228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 14:39:43.696228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:43.696228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 349/716] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:43 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:43.703247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826b34f-c641-40b6-b161-616d88fd8a4e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:43.745246 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95c2e0-0e3a-444c-b822-9e491e8b9d93 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:43.768693 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:43.835924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1826b34f-c641-40b6-b161-616d88fd8a4e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:43.838505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eaa325-8560-4f29-b176-109c4b7dfcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:43.839145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eaa325-8560-4f29-b176-109c4b7dfcef None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:43.839423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eaa325-8560-4f29-b176-109c4b7dfcef None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:43.839696 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eaa325-8560-4f29-b176-109c4b7dfcef None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:43.840371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eaa325-8560-4f29-b176-109c4b7dfcef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:43.840675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eaa325-8560-4f29-b176-109c4b7dfcef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:43.842644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eaa325-8560-4f29-b176-109c4b7dfcef None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:43.861569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 368/717] 10.222.0.60 () {56 vars in 1124 bytes} [Mon Sep 19 14:39:43 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:43.900250 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195332f-764a-48df-b725-96cfefcd744b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:43.900938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195332f-764a-48df-b725-96cfefcd744b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:43.901281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195332f-764a-48df-b725-96cfefcd744b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:43.901631 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195332f-764a-48df-b725-96cfefcd744b None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:43.902279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195332f-764a-48df-b725-96cfefcd744b None admin] RBAC: Adding query filter params (name=network) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:43.902581 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195332f-764a-48df-b725-96cfefcd744b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:43.904623 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195332f-764a-48df-b725-96cfefcd744b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:43.915960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 350/718] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Sep 19 14:39:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:43.922179 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eaa325-8560-4f29-b176-109c4b7dfcef None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:44.093071 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:44.176043 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eaa325-8560-4f29-b176-109c4b7dfcef None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:44.224556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1acd291-290e-4267-8427-5d18eef3b383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:44.225174 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acd291-290e-4267-8427-5d18eef3b383 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:44.225452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acd291-290e-4267-8427-5d18eef3b383 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:44.225725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1acd291-290e-4267-8427-5d18eef3b383 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:44.226351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1acd291-290e-4267-8427-5d18eef3b383 None admin] RBAC: Adding query filter params (type=network) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:44.226652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1acd291-290e-4267-8427-5d18eef3b383 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:44.228045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1acd291-290e-4267-8427-5d18eef3b383 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:44.233734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 369/719] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Sep 19 14:39:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:45.516151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2d222-95af-4ff4-87f3-feea9929b0ea None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:45.516151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2d222-95af-4ff4-87f3-feea9929b0ea None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:45.516151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2d222-95af-4ff4-87f3-feea9929b0ea None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:45.517229 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 351/720] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:45.522887 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961dd66a-08ee-4ce6-9ceb-b351f9010fb8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:45.523294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961dd66a-08ee-4ce6-9ceb-b351f9010fb8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:45.523466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961dd66a-08ee-4ce6-9ceb-b351f9010fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:45.548808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-961dd66a-08ee-4ce6-9ceb-b351f9010fb8 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:45.550266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-961dd66a-08ee-4ce6-9ceb-b351f9010fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:45.631560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 370/721] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:45.799412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab7553-8b6b-4198-92d8-2d0966bdb827 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:45.799591 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab7553-8b6b-4198-92d8-2d0966bdb827 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:45.799903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab7553-8b6b-4198-92d8-2d0966bdb827 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:45.801481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 352/722] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:45.806782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1409614-76bd-4174-8be0-d74eee24a4f8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:45.807072 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1409614-76bd-4174-8be0-d74eee24a4f8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:45.807352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1409614-76bd-4174-8be0-d74eee24a4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:45.830244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b1409614-76bd-4174-8be0-d74eee24a4f8 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:45.831432 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b1409614-76bd-4174-8be0-d74eee24a4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:45.840555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1b6bf-ccca-4f1d-983e-e95952915afc None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:45.840831 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1b6bf-ccca-4f1d-983e-e95952915afc None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:45.841654 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1b6bf-ccca-4f1d-983e-e95952915afc None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:45.843320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 353/723] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:45.848001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbf07d-d76a-46f4-9974-55d58a7eb01c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:45.848289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbf07d-d76a-46f4-9974-55d58a7eb01c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:45.848565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dbf07d-d76a-46f4-9974-55d58a7eb01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:45.884922 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b3dbf07d-d76a-46f4-9974-55d58a7eb01c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:45.886157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b3dbf07d-d76a-46f4-9974-55d58a7eb01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:45.888695 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:45.914440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 371/724] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:45.997943 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 354/725] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:46.018374 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1409614-76bd-4174-8be0-d74eee24a4f8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:46.142420 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1409614-76bd-4174-8be0-d74eee24a4f8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:46.183357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ccb5f-785b-4ca5-8751-207513d42e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:46.184022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ccb5f-785b-4ca5-8751-207513d42e0c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:46.184305 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ccb5f-785b-4ca5-8751-207513d42e0c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:46.184581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ccb5f-785b-4ca5-8751-207513d42e0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:46.201613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ccb5f-785b-4ca5-8751-207513d42e0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-869ccb5f-785b-4ca5-8751-207513d42e0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:46.206454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:46.207429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:46.208349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:46.208349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:46.208349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:46.208349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:46.208349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:46.208349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:46.208349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:46.208349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:46.208349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:46.208349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:46.208349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:46.208349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:46.213368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 372/726] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:39:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:46.218809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbf07d-d76a-46f4-9974-55d58a7eb01c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:46.235846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ccb5f-785b-4ca5-8751-207513d42e0c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:46.297747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:46.307950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:46.397720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dbf07d-d76a-46f4-9974-55d58a7eb01c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:46.411394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ccb5f-785b-4ca5-8751-207513d42e0c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:46.444529 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006151d-52c6-4588-be41-b8342f3af2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:46.445399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006151d-52c6-4588-be41-b8342f3af2d5 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:46.446090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006151d-52c6-4588-be41-b8342f3af2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:46.446438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006151d-52c6-4588-be41-b8342f3af2d5 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:46.447393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006151d-52c6-4588-be41-b8342f3af2d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:46.447699 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006151d-52c6-4588-be41-b8342f3af2d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:46.450026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006151d-52c6-4588-be41-b8342f3af2d5 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:46.454180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d21acf-64c6-4604-bf84-5d351d1a7f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:46.454818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d21acf-64c6-4604-bf84-5d351d1a7f69 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:46.455102 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d21acf-64c6-4604-bf84-5d351d1a7f69 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:46.455376 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d21acf-64c6-4604-bf84-5d351d1a7f69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:46.461546 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 355/727] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:46.462475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006151d-52c6-4588-be41-b8342f3af2d5 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:46.472581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d21acf-64c6-4604-bf84-5d351d1a7f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-73d21acf-64c6-4604-bf84-5d351d1a7f69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:46.479558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:46.480569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:46.481431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:46.481431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:46.481431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:46.481431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:46.481431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:46.481431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:46.481431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:46.481431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:46.481431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:46.481431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:46.481431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:46.481431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:46.483219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 373/728] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:39:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:39:46.483219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d21acf-64c6-4604-bf84-5d351d1a7f69 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:46.577483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006151d-52c6-4588-be41-b8342f3af2d5 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:46.587030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:46.614448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d21acf-64c6-4604-bf84-5d351d1a7f69 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:46.616210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:46.624636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be296ccc-1504-4644-9776-2650aa25d419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:46.625292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be296ccc-1504-4644-9776-2650aa25d419 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:46.625574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be296ccc-1504-4644-9776-2650aa25d419 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:46.625849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be296ccc-1504-4644-9776-2650aa25d419 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:46.637496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be296ccc-1504-4644-9776-2650aa25d419 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-be296ccc-1504-4644-9776-2650aa25d419 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:46.644025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:46.644998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:46.645848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:46.645848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:46.645848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:46.645848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:46.645848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:46.645848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:46.645848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:46.645848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:46.645848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:46.645848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:46.645848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:46.645848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:46.650104 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 356/729] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:46.650925 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be296ccc-1504-4644-9776-2650aa25d419 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:46.662882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add7754-897e-4408-a632-3daafb846f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:46.663951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add7754-897e-4408-a632-3daafb846f5c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:46.664721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add7754-897e-4408-a632-3daafb846f5c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:46.665452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add7754-897e-4408-a632-3daafb846f5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:46.678818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add7754-897e-4408-a632-3daafb846f5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:46.681153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add7754-897e-4408-a632-3daafb846f5c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-2add7754-897e-4408-a632-3daafb846f5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:46.706932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:46.708578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:46.709481 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:46.709481 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:46.709481 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:46.709481 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 374/730] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:46.710392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add7754-897e-4408-a632-3daafb846f5c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:46.772878 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be296ccc-1504-4644-9776-2650aa25d419 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:46.820714 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6868e-0673-4281-a02c-1715a35ecddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:46.822024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6868e-0673-4281-a02c-1715a35ecddf None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:46.822730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6868e-0673-4281-a02c-1715a35ecddf None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:46.823158 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6868e-0673-4281-a02c-1715a35ecddf None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:46.824230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6868e-0673-4281-a02c-1715a35ecddf None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:46.825081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6868e-0673-4281-a02c-1715a35ecddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:46.826954 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6868e-0673-4281-a02c-1715a35ecddf None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:46.831384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add7754-897e-4408-a632-3daafb846f5c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:46.838183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 357/731] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:46.839353 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6868e-0673-4281-a02c-1715a35ecddf None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:46.875948 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9f383-7b09-45b6-89cc-96e33e320137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:46.876511 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9f383-7b09-45b6-89cc-96e33e320137 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:46.876929 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9f383-7b09-45b6-89cc-96e33e320137 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:46.877351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9f383-7b09-45b6-89cc-96e33e320137 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:46.878195 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9f383-7b09-45b6-89cc-96e33e320137 None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:46.878609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9f383-7b09-45b6-89cc-96e33e320137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:46.880300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9f383-7b09-45b6-89cc-96e33e320137 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:46.880817 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:46.898403 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 375/732] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:46.899593 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9f383-7b09-45b6-89cc-96e33e320137 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:46.921902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:46.985846 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6868e-0673-4281-a02c-1715a35ecddf None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:47.022455 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9f383-7b09-45b6-89cc-96e33e320137 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:47.024306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837f137-2e1d-455c-8f1e-e017ad512972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:47.025411 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837f137-2e1d-455c-8f1e-e017ad512972 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:47.025411 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837f137-2e1d-455c-8f1e-e017ad512972 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:47.025619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b837f137-2e1d-455c-8f1e-e017ad512972 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:47.026079 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837f137-2e1d-455c-8f1e-e017ad512972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:47.026380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837f137-2e1d-455c-8f1e-e017ad512972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:47.027751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b837f137-2e1d-455c-8f1e-e017ad512972 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:47.041541 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 358/733] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:47.042246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837f137-2e1d-455c-8f1e-e017ad512972 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:47.060499 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59780a-7671-4c07-8567-53c6927fc965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:47.061163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59780a-7671-4c07-8567-53c6927fc965 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:47.061444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59780a-7671-4c07-8567-53c6927fc965 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:47.061718 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59780a-7671-4c07-8567-53c6927fc965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:47.074334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59780a-7671-4c07-8567-53c6927fc965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:47.076461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59780a-7671-4c07-8567-53c6927fc965 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-7a59780a-7671-4c07-8567-53c6927fc965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:47.095587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:47.096592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:47.097503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:47.097503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:47.097503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:47.097503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 376/734] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:47.098244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a25634-be5b-4477-b71b-80b93e2f06c7 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:47.098411 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a25634-be5b-4477-b71b-80b93e2f06c7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:47.098745 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a25634-be5b-4477-b71b-80b93e2f06c7 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:47.100422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 377/735] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:47.101143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a25634-be5b-4477-b71b-80b93e2f06c7 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:47.151221 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:47.165220 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b837f137-2e1d-455c-8f1e-e017ad512972 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:47.173938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:47.205353 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fef0c-4535-4abe-8383-1e58a8eff000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:47.205987 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fef0c-4535-4abe-8383-1e58a8eff000 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:47.206299 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fef0c-4535-4abe-8383-1e58a8eff000 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:47.206578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fef0c-4535-4abe-8383-1e58a8eff000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:47.210138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a25634-be5b-4477-b71b-80b93e2f06c7 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:47.217417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fef0c-4535-4abe-8383-1e58a8eff000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:47.218866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fef0c-4535-4abe-8383-1e58a8eff000 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-636fef0c-4535-4abe-8383-1e58a8eff000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:47.233901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:47.235011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:47.235998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:47.235998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:47.235998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:47.235998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 359/736] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:47.235998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fef0c-4535-4abe-8383-1e58a8eff000 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:47.246718 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4fd809-23b6-4b79-af07-32dd208b0d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:47.248016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fd809-23b6-4b79-af07-32dd208b0d6e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:47.248302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fd809-23b6-4b79-af07-32dd208b0d6e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:47.248578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4fd809-23b6-4b79-af07-32dd208b0d6e None admin] PATH_INFO: `/v3/users/nova` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:47.256134 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4fd809-23b6-4b79-af07-32dd208b0d6e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:47.257559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4fd809-23b6-4b79-af07-32dd208b0d6e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-9a4fd809-23b6-4b79-af07-32dd208b0d6e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:47.268609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:47.269740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:47.270700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:47.270700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:47.270700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:47.270700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:47.270700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:47.270700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:47.270700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:47.270700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:47.270700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:39:47.270700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:47.270700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 378/737] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Sep 19 14:39:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:47.271312 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade4530-327b-44a3-81c4-47c9164fd71b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:47.271312 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade4530-327b-44a3-81c4-47c9164fd71b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:47.271312 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ade4530-327b-44a3-81c4-47c9164fd71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:47.288825 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8ade4530-327b-44a3-81c4-47c9164fd71b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:47.290015 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8ade4530-327b-44a3-81c4-47c9164fd71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:47.337790 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fef0c-4535-4abe-8383-1e58a8eff000 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:47.359132 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 379/738] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:47.360519 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfbf5e-64bb-4b29-ab7a-0f161e6fec07 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:47.360863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfbf5e-64bb-4b29-ab7a-0f161e6fec07 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:47.361135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfbf5e-64bb-4b29-ab7a-0f161e6fec07 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:47.363951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 380/739] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:47.364992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfbf5e-64bb-4b29-ab7a-0f161e6fec07 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:47.373650 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7feae-eb86-4885-98b6-22857a8ef1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:47.374254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7feae-eb86-4885-98b6-22857a8ef1cd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:47.374529 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7feae-eb86-4885-98b6-22857a8ef1cd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:47.374798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7feae-eb86-4885-98b6-22857a8ef1cd None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:47.375434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7feae-eb86-4885-98b6-22857a8ef1cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:47.375722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7feae-eb86-4885-98b6-22857a8ef1cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:47.377135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7feae-eb86-4885-98b6-22857a8ef1cd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:47.387178 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:47.395742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 360/740] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:47.396702 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7feae-eb86-4885-98b6-22857a8ef1cd None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:47.442549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:47.496072 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfbf5e-64bb-4b29-ab7a-0f161e6fec07 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:47.503192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7feae-eb86-4885-98b6-22857a8ef1cd None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:47.532276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731463c0-6a0d-4185-89d4-dacaf84b23ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:47.532886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731463c0-6a0d-4185-89d4-dacaf84b23ab None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:47.533332 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731463c0-6a0d-4185-89d4-dacaf84b23ab None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:47.533613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731463c0-6a0d-4185-89d4-dacaf84b23ab None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:47.534828 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731463c0-6a0d-4185-89d4-dacaf84b23ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:47.535143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731463c0-6a0d-4185-89d4-dacaf84b23ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:47.536518 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731463c0-6a0d-4185-89d4-dacaf84b23ab None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:47.538629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d2566-31f3-41d3-8da2-7bc899bb1333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:47.539238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d2566-31f3-41d3-8da2-7bc899bb1333 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:47.539522 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d2566-31f3-41d3-8da2-7bc899bb1333 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:47.539798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d2566-31f3-41d3-8da2-7bc899bb1333 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:47.540550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d2566-31f3-41d3-8da2-7bc899bb1333 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:47.540858 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d2566-31f3-41d3-8da2-7bc899bb1333 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:47.542274 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d2566-31f3-41d3-8da2-7bc899bb1333 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:47.553070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 381/741] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:47.553070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a27f85-52c7-44ef-9bb3-1adbfa9a759b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:47.553070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a27f85-52c7-44ef-9bb3-1adbfa9a759b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:47.553070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a27f85-52c7-44ef-9bb3-1adbfa9a759b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:47.565579 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 361/742] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Sep 19 14:39:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:47.570272 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d2566-31f3-41d3-8da2-7bc899bb1333 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:47.574647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-89a27f85-52c7-44ef-9bb3-1adbfa9a759b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:47.575998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-89a27f85-52c7-44ef-9bb3-1adbfa9a759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:47.631653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:47.651348 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 382/743] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 19 14:39:47.652050 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a27f85-52c7-44ef-9bb3-1adbfa9a759b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:47.685640 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d2566-31f3-41d3-8da2-7bc899bb1333 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:47.703955 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:47.723672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6c8f3-bdd1-4f75-9c31-e522c140b3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:47.724323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6c8f3-bdd1-4f75-9c31-e522c140b3ee None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:47.724601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6c8f3-bdd1-4f75-9c31-e522c140b3ee None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:47.724867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6c8f3-bdd1-4f75-9c31-e522c140b3ee None admin] PATH_INFO: `/v3/users/swift` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:47.732383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6c8f3-bdd1-4f75-9c31-e522c140b3ee None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:47.733817 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de6c8f3-bdd1-4f75-9c31-e522c140b3ee None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-6de6c8f3-bdd1-4f75-9c31-e522c140b3ee None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:47.745735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:47.746691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:47.747731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:47.747731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:47.747731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:47.747731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:47.747731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:47.747731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:47.747731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:47.747731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:47.747731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 14:39:47.747731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:47.751795 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 362/744] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:39:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:47.752500 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6c8f3-bdd1-4f75-9c31-e522c140b3ee None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:47.760375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a27f85-52c7-44ef-9bb3-1adbfa9a759b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:47.798296 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d44400-6ecd-411a-9f28-7332005cbe50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:47.798919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d44400-6ecd-411a-9f28-7332005cbe50 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:47.799464 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d44400-6ecd-411a-9f28-7332005cbe50 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:47.799737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d44400-6ecd-411a-9f28-7332005cbe50 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:47.807111 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d44400-6ecd-411a-9f28-7332005cbe50 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:47.808676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d44400-6ecd-411a-9f28-7332005cbe50 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-d2d44400-6ecd-411a-9f28-7332005cbe50 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:47.822535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:47.823557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:47.825141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:47.825141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:47.825141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:47.825141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:47.825141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:47.825141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:47.825141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:47.825141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:47.825141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 14:39:47.825141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:47.829499 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 383/745] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Sep 19 14:39:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:47.829499 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d44400-6ecd-411a-9f28-7332005cbe50 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:47.853525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6c8f3-bdd1-4f75-9c31-e522c140b3ee None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:47.871239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:47.891741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4532098-6d9e-4c48-a096-f0d282423cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:47.892406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4532098-6d9e-4c48-a096-f0d282423cc2 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:47.892690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4532098-6d9e-4c48-a096-f0d282423cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:47.892967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4532098-6d9e-4c48-a096-f0d282423cc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:47.904737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4532098-6d9e-4c48-a096-f0d282423cc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:47.906208 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4532098-6d9e-4c48-a096-f0d282423cc2 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c4532098-6d9e-4c48-a096-f0d282423cc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:47.920794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:47.921816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:47.922723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:47.922723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:47.922723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:47.922723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 363/746] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:47.922961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4532098-6d9e-4c48-a096-f0d282423cc2 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:47.946132 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d44400-6ecd-411a-9f28-7332005cbe50 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:47.955595 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:47.983513 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c89b87-2507-4175-8857-04df1bdaa554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:47.984168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c89b87-2507-4175-8857-04df1bdaa554 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:47.984457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c89b87-2507-4175-8857-04df1bdaa554 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:47.984725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c89b87-2507-4175-8857-04df1bdaa554 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:47.985378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c89b87-2507-4175-8857-04df1bdaa554 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:47.986738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c89b87-2507-4175-8857-04df1bdaa554 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:48.008173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 384/747] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Sep 19 14:39:47 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:39:48.009001 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c89b87-2507-4175-8857-04df1bdaa554 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:48.028279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4532098-6d9e-4c48-a096-f0d282423cc2 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:48.077737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c09a7-8542-4bb0-92a7-9a1c8dca1086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:48.078372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c09a7-8542-4bb0-92a7-9a1c8dca1086 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:48.078655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c09a7-8542-4bb0-92a7-9a1c8dca1086 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:48.078929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c09a7-8542-4bb0-92a7-9a1c8dca1086 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:48.079800 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c09a7-8542-4bb0-92a7-9a1c8dca1086 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:48.080161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c09a7-8542-4bb0-92a7-9a1c8dca1086 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:48.081805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c09a7-8542-4bb0-92a7-9a1c8dca1086 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:48.105251 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 364/748] 10.222.0.60 () {56 vars in 1135 bytes} [Mon Sep 19 14:39:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:48.105980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c09a7-8542-4bb0-92a7-9a1c8dca1086 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:48.137777 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c89b87-2507-4175-8857-04df1bdaa554 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:48.144887 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:48.176829 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e73081-82e2-410f-ad0c-a5d2a3bfe7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:48.177457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e73081-82e2-410f-ad0c-a5d2a3bfe7fc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:48.177739 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e73081-82e2-410f-ad0c-a5d2a3bfe7fc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:48.178011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e73081-82e2-410f-ad0c-a5d2a3bfe7fc None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:48.178747 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e73081-82e2-410f-ad0c-a5d2a3bfe7fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:48.179053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e73081-82e2-410f-ad0c-a5d2a3bfe7fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:48.181447 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e73081-82e2-410f-ad0c-a5d2a3bfe7fc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:48.203749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 385/749] 10.222.0.60 () {56 vars in 1149 bytes} [Mon Sep 19 14:39:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:48.205992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e73081-82e2-410f-ad0c-a5d2a3bfe7fc None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:48.239173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:48.259810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c09a7-8542-4bb0-92a7-9a1c8dca1086 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:48.299901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e7cae-a080-448b-ab08-a16782f04cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:48.300522 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e7cae-a080-448b-ab08-a16782f04cc9 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:48.300811 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e7cae-a080-448b-ab08-a16782f04cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:48.301084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4e7cae-a080-448b-ab08-a16782f04cc9 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:48.301745 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e7cae-a080-448b-ab08-a16782f04cc9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:48.303147 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4e7cae-a080-448b-ab08-a16782f04cc9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:48.320388 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 365/750] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Sep 19 14:39:48 2022] POST /identity/v3/services => generated 240 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:39:48.321241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e7cae-a080-448b-ab08-a16782f04cc9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:48.340623 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e73081-82e2-410f-ad0c-a5d2a3bfe7fc None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:48.396812 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43dc75-775f-42ee-83a4-7dcc62d25753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:48.397423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43dc75-775f-42ee-83a4-7dcc62d25753 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:48.397708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43dc75-775f-42ee-83a4-7dcc62d25753 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:48.397986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c43dc75-775f-42ee-83a4-7dcc62d25753 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:48.398693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43dc75-775f-42ee-83a4-7dcc62d25753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:48.399235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43dc75-775f-42ee-83a4-7dcc62d25753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:48.400531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c43dc75-775f-42ee-83a4-7dcc62d25753 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:48.418542 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 386/751] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:48.418542 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43dc75-775f-42ee-83a4-7dcc62d25753 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:48.455245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:48.482559 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4e7cae-a080-448b-ab08-a16782f04cc9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:48.521474 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:48.534645 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e8673-baaa-4e4e-820f-cf61f116d738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:48.535490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e8673-baaa-4e4e-820f-cf61f116d738 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:48.535950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e8673-baaa-4e4e-820f-cf61f116d738 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:48.536368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e8673-baaa-4e4e-820f-cf61f116d738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:48.542547 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c43dc75-775f-42ee-83a4-7dcc62d25753 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:48.587294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e72326-fae4-460b-b650-487b323f1ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:48.587981 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e72326-fae4-460b-b650-487b323f1ac2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:48.588268 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e72326-fae4-460b-b650-487b323f1ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:48.588542 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e72326-fae4-460b-b650-487b323f1ac2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:48.625024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e8673-baaa-4e4e-820f-cf61f116d738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:48.627458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e8673-baaa-4e4e-820f-cf61f116d738 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:48.660666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e72326-fae4-460b-b650-487b323f1ac2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:48.662120 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e72326-fae4-460b-b650-487b323f1ac2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:48.737383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:48.758216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 366/752] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:39:48.759102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e8673-baaa-4e4e-820f-cf61f116d738 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:48.765300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:48.790093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 387/753] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:39:48.790957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e72326-fae4-460b-b650-487b323f1ac2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:48.878461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e8673-baaa-4e4e-820f-cf61f116d738 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:48.913214 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e72326-fae4-460b-b650-487b323f1ac2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:48.923220 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21b867-d14b-4722-a922-8feec98d1175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:48.924705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21b867-d14b-4722-a922-8feec98d1175 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:48.925078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21b867-d14b-4722-a922-8feec98d1175 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:48.925422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21b867-d14b-4722-a922-8feec98d1175 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:48.936291 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21b867-d14b-4722-a922-8feec98d1175 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:48.938663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21b867-d14b-4722-a922-8feec98d1175 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:48.953203 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e269853-8df4-4d63-b6d6-b1f81881e982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:48.953839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e269853-8df4-4d63-b6d6-b1f81881e982 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:48.954121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e269853-8df4-4d63-b6d6-b1f81881e982 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:48.954395 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e269853-8df4-4d63-b6d6-b1f81881e982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:48.954498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-ad21b867-d14b-4722-a922-8feec98d1175 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:48.954498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:48.954498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:48.954498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:48.954498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:48.954498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:48.954498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:48.954498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:48.954498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:48.954498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:48.954498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:48.954498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:48.954498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:48.954498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:48.954498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:48.955321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:48.956278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:48.956278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:48.956278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:48.956278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:48.956278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:48.956278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:48.956278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:48.956278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:48.956278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 367/754] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:48.956773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21b867-d14b-4722-a922-8feec98d1175 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:48.970075 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e269853-8df4-4d63-b6d6-b1f81881e982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:48.971557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e269853-8df4-4d63-b6d6-b1f81881e982 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-9e269853-8df4-4d63-b6d6-b1f81881e982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:48.987871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:48.988833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:48.989751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:48.989751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:48.989751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:48.989751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 388/755] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:48.993885 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e269853-8df4-4d63-b6d6-b1f81881e982 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:49.014340 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:49.033814 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:49.070643 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21b867-d14b-4722-a922-8feec98d1175 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:49.106118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e385c-7906-4d05-9667-3809718c77e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:49.106843 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e385c-7906-4d05-9667-3809718c77e7 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:49.107124 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e385c-7906-4d05-9667-3809718c77e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:49.107396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e385c-7906-4d05-9667-3809718c77e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:49.114212 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e269853-8df4-4d63-b6d6-b1f81881e982 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:49.123261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e385c-7906-4d05-9667-3809718c77e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:49.124699 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e385c-7906-4d05-9667-3809718c77e7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-893e385c-7906-4d05-9667-3809718c77e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:49.139520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:49.140630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:49.141562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:49.141562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:49.141562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:49.141562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 368/756] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:49.141794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e385c-7906-4d05-9667-3809718c77e7 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:49.154458 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7f9f2-11eb-4a27-873d-089a1cd38e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:49.155555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7f9f2-11eb-4a27-873d-089a1cd38e03 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:49.156072 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7f9f2-11eb-4a27-873d-089a1cd38e03 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:49.156550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7f9f2-11eb-4a27-873d-089a1cd38e03 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:49.157461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7f9f2-11eb-4a27-873d-089a1cd38e03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:49.158088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7f9f2-11eb-4a27-873d-089a1cd38e03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:49.159898 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7f9f2-11eb-4a27-873d-089a1cd38e03 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:49.174544 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 389/757] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:49.175884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7f9f2-11eb-4a27-873d-089a1cd38e03 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:49.242930 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:49.248897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e385c-7906-4d05-9667-3809718c77e7 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:49.285770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a8de8-91c2-4a96-b387-bb489f5e686f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:49.286389 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a8de8-91c2-4a96-b387-bb489f5e686f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:49.286673 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a8de8-91c2-4a96-b387-bb489f5e686f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:49.286949 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66a8de8-91c2-4a96-b387-bb489f5e686f None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:49.287594 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a8de8-91c2-4a96-b387-bb489f5e686f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:49.287925 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a8de8-91c2-4a96-b387-bb489f5e686f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:49.289396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66a8de8-91c2-4a96-b387-bb489f5e686f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:49.295860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7f9f2-11eb-4a27-873d-089a1cd38e03 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:49.299706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:49.302043 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 369/758] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:49.302860 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a8de8-91c2-4a96-b387-bb489f5e686f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:49.346262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbb10e-6a9f-4106-98d0-88c6e7843ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:49.347330 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbb10e-6a9f-4106-98d0-88c6e7843ad8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:49.348424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbb10e-6a9f-4106-98d0-88c6e7843ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:49.348990 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afbb10e-6a9f-4106-98d0-88c6e7843ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:49.350560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbb10e-6a9f-4106-98d0-88c6e7843ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:49.351224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbb10e-6a9f-4106-98d0-88c6e7843ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:49.353267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afbb10e-6a9f-4106-98d0-88c6e7843ad8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:49.371943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 390/759] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:49.373781 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbb10e-6a9f-4106-98d0-88c6e7843ad8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:49.415129 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66a8de8-91c2-4a96-b387-bb489f5e686f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:49.452613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed021728-8b54-427b-abc9-f9bb0c60151f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:49.453273 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed021728-8b54-427b-abc9-f9bb0c60151f None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:49.453552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed021728-8b54-427b-abc9-f9bb0c60151f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:49.453825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed021728-8b54-427b-abc9-f9bb0c60151f None admin] PATH_INFO: `/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/59b5d2fd362b417c9b9be75f88a667ad/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:49.455736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed021728-8b54-427b-abc9-f9bb0c60151f None admin] RBAC: Authorizing `identity:create_grant(project_id=7dee98edf19d478ca4f746ff85579fba, user_id=59b5d2fd362b417c9b9be75f88a667ad, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:49.457280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed021728-8b54-427b-abc9-f9bb0c60151f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:49.474396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 370/760] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:39:49 2022] PUT /identity/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/59b5d2fd362b417c9b9be75f88a667ad/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:39:49.475232 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed021728-8b54-427b-abc9-f9bb0c60151f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:49.500597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:49.519169 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afbb10e-6a9f-4106-98d0-88c6e7843ad8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:49.560713 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3e29c-a45a-436a-b1ec-dae00552aa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:49.561940 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3e29c-a45a-436a-b1ec-dae00552aa82 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:49.562741 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3e29c-a45a-436a-b1ec-dae00552aa82 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:49.563472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e3e29c-a45a-436a-b1ec-dae00552aa82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:49.572324 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3e29c-a45a-436a-b1ec-dae00552aa82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:49.574210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e3e29c-a45a-436a-b1ec-dae00552aa82 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:49.590604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-67e3e29c-a45a-436a-b1ec-dae00552aa82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:49.590604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:49.590604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:49.590604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:49.590604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:49.590604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:49.590604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:49.590604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:49.590604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:49.590604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:49.590604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:49.590604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:49.590604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:49.590604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:49.590604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:49.592276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:49.593208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:49.593208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:49.593208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:49.593208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:49.593208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:49.593208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:49.593208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:49.593208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:49.596012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 391/761] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:49.601285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3e29c-a45a-436a-b1ec-dae00552aa82 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:49.635121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:49.641010 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed021728-8b54-427b-abc9-f9bb0c60151f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:49.720442 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb19630-a7b6-4b08-b6fa-235007cec304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:49.721083 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb19630-a7b6-4b08-b6fa-235007cec304 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:49.721366 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb19630-a7b6-4b08-b6fa-235007cec304 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:49.721640 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb19630-a7b6-4b08-b6fa-235007cec304 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:49.730034 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb19630-a7b6-4b08-b6fa-235007cec304 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:49.731680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb19630-a7b6-4b08-b6fa-235007cec304 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:49.744596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-1cb19630-a7b6-4b08-b6fa-235007cec304 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:49.744596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:49.744596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:49.744596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:49.744596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:49.744596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:49.744596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:49.744596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:49.744596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:49.744596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:49.744596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:49.744596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:49.744596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:49.744596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:49.744596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:49.746018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:49.746948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:49.746948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:49.746948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:49.746948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:49.746948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:49.746948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:49.746948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:49.746948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:49.746948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 371/762] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:49.753241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb19630-a7b6-4b08-b6fa-235007cec304 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:49.783061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e3e29c-a45a-436a-b1ec-dae00552aa82 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:49.826544 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e105ff4-191e-4b29-b031-a24518f197e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:49.828195 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e105ff4-191e-4b29-b031-a24518f197e7 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:49.828668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e105ff4-191e-4b29-b031-a24518f197e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:49.829126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e105ff4-191e-4b29-b031-a24518f197e7 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:49.830195 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e105ff4-191e-4b29-b031-a24518f197e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:49.830511 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e105ff4-191e-4b29-b031-a24518f197e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:49.831991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e105ff4-191e-4b29-b031-a24518f197e7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:49.848676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:49.849468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 392/763] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:49.854588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e105ff4-191e-4b29-b031-a24518f197e7 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:49.909410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb19630-a7b6-4b08-b6fa-235007cec304 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:49.939408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:49.970933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e105ff4-191e-4b29-b031-a24518f197e7 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:49.982921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b88546-7aab-4ed4-b025-83df54fd8bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:49.983548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b88546-7aab-4ed4-b025-83df54fd8bc1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:49.983850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b88546-7aab-4ed4-b025-83df54fd8bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:49.984137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b88546-7aab-4ed4-b025-83df54fd8bc1 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:49.984935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b88546-7aab-4ed4-b025-83df54fd8bc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:49.985245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b88546-7aab-4ed4-b025-83df54fd8bc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:49.986866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b88546-7aab-4ed4-b025-83df54fd8bc1 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:50.012055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37673a14-3514-46cc-af12-f40f7bfb1993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:50.013070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37673a14-3514-46cc-af12-f40f7bfb1993 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:50.013557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37673a14-3514-46cc-af12-f40f7bfb1993 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:50.014075 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37673a14-3514-46cc-af12-f40f7bfb1993 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:50.014993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37673a14-3514-46cc-af12-f40f7bfb1993 None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=bfc14d32f4314c6e93d20a9c5a45c220) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:50.015552 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37673a14-3514-46cc-af12-f40f7bfb1993 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:50.017239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37673a14-3514-46cc-af12-f40f7bfb1993 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:50.018016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 372/764] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:50.023767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b88546-7aab-4ed4-b025-83df54fd8bc1 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:50.024628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 393/765] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:39:49 2022] GET /identity/v3/role_assignments?user.id=bfc14d32f4314c6e93d20a9c5a45c220&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:39:50.151877 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b88546-7aab-4ed4-b025-83df54fd8bc1 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:50.202117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367764f-d293-471f-9895-c13e64c90b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:50.202754 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367764f-d293-471f-9895-c13e64c90b38 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:50.203039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367764f-d293-471f-9895-c13e64c90b38 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:50.203321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367764f-d293-471f-9895-c13e64c90b38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:50.204090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367764f-d293-471f-9895-c13e64c90b38 None admin] RBAC: Adding query filter params (role.id=15fbc4d519f64f4cacecc3b20fe826ae, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=0fefb7fa6f484341a19a1a8bee31b116) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:50.204403 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367764f-d293-471f-9895-c13e64c90b38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:50.205871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367764f-d293-471f-9895-c13e64c90b38 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:50.211473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:50.214935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 373/766] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:39:50 2022] GET /identity/v3/role_assignments?user.id=0fefb7fa6f484341a19a1a8bee31b116&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=15fbc4d519f64f4cacecc3b20fe826ae => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:50.595763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844b45e-56e8-4501-ae75-b2fd613b824c None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:50.596128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844b45e-56e8-4501-ae75-b2fd613b824c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:50.596471 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844b45e-56e8-4501-ae75-b2fd613b824c None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:50.598060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 394/767] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:50.603458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870c180-8039-4857-a660-7e7d2a426df8 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:50.603753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870c180-8039-4857-a660-7e7d2a426df8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:50.604124 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870c180-8039-4857-a660-7e7d2a426df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:50.632923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6870c180-8039-4857-a660-7e7d2a426df8 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:50.634142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6870c180-8039-4857-a660-7e7d2a426df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:50.713565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 374/768] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:39:50.888444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a46fd-2ab5-4154-94e6-b7a684a323ab None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:50.888959 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a46fd-2ab5-4154-94e6-b7a684a323ab None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:50.888959 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a46fd-2ab5-4154-94e6-b7a684a323ab None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:50.890452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 395/769] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:50.896039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef23ef-d138-4d01-b1fa-458852060504 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:50.896457 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef23ef-d138-4d01-b1fa-458852060504 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:50.896651 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef23ef-d138-4d01-b1fa-458852060504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:50.925098 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1bef23ef-d138-4d01-b1fa-458852060504 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:50.926299 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1bef23ef-d138-4d01-b1fa-458852060504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:51.008612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 375/770] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:39:51.103906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a46fd-2ab5-4154-94e6-b7a684a323ab None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:51.244219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94a46fd-2ab5-4154-94e6-b7a684a323ab None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:51.289835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747014cf-137a-402a-810b-3af96667c67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:51.290461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747014cf-137a-402a-810b-3af96667c67e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:51.290734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747014cf-137a-402a-810b-3af96667c67e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:51.291002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747014cf-137a-402a-810b-3af96667c67e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:51.291631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747014cf-137a-402a-810b-3af96667c67e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:51.293011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747014cf-137a-402a-810b-3af96667c67e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-747014cf-137a-402a-810b-3af96667c67e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:51.306409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:51.307462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:51.308375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:51.308375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:51.308375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:51.308375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:51.308375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:51.308375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:51.308375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:51.308375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:51.308375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:51.308375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:51.308375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:39:51.308375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:39:51.308375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 14:39:51.308375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:51.308375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 396/771] 10.222.0.60 () {56 vars in 1129 bytes} [Mon Sep 19 14:39:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:51.314234 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef23ef-d138-4d01-b1fa-458852060504 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:51.376411 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:51.431958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef23ef-d138-4d01-b1fa-458852060504 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:51.474013 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acc9c7-7846-41af-a9f0-612c359589da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:51.474623 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acc9c7-7846-41af-a9f0-612c359589da None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:51.474997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acc9c7-7846-41af-a9f0-612c359589da None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:51.475277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09acc9c7-7846-41af-a9f0-612c359589da None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:51.475918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09acc9c7-7846-41af-a9f0-612c359589da None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:51.476269 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09acc9c7-7846-41af-a9f0-612c359589da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:51.477633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09acc9c7-7846-41af-a9f0-612c359589da None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:51.485011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 376/772] 10.222.0.60 () {56 vars in 1124 bytes} [Mon Sep 19 14:39:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:51.488641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747014cf-137a-402a-810b-3af96667c67e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:51.492044 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:51.616555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747014cf-137a-402a-810b-3af96667c67e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:51.632869 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acc9c7-7846-41af-a9f0-612c359589da None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:51.688042 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bebd11-f4db-4614-a0c1-c435fca8a46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:51.688661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bebd11-f4db-4614-a0c1-c435fca8a46e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:51.688934 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bebd11-f4db-4614-a0c1-c435fca8a46e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:51.689201 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bebd11-f4db-4614-a0c1-c435fca8a46e None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:51.689826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bebd11-f4db-4614-a0c1-c435fca8a46e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:51.690124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bebd11-f4db-4614-a0c1-c435fca8a46e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:51.691481 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bebd11-f4db-4614-a0c1-c435fca8a46e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:51.703940 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 397/773] 10.222.0.60 () {56 vars in 1124 bytes} [Mon Sep 19 14:39:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:51.703940 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bebd11-f4db-4614-a0c1-c435fca8a46e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:51.787625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:51.831343 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bebd11-f4db-4614-a0c1-c435fca8a46e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:51.865356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09acc9c7-7846-41af-a9f0-612c359589da None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:51.868878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830b622-4d35-4601-a5d3-2321c6793b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:51.869436 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830b622-4d35-4601-a5d3-2321c6793b05 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:51.869713 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830b622-4d35-4601-a5d3-2321c6793b05 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:51.869983 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830b622-4d35-4601-a5d3-2321c6793b05 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:51.870782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830b622-4d35-4601-a5d3-2321c6793b05 None admin] RBAC: Adding query filter params (interface=public, service_id=92051670864744838a8ee22969c5aeb8, region_id=RegionOne) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:51.871086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830b622-4d35-4601-a5d3-2321c6793b05 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:51.873045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3830b622-4d35-4601-a5d3-2321c6793b05 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:51.887897 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 398/774] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Sep 19 14:39:51 2022] GET /identity/v3/endpoints?service_id=92051670864744838a8ee22969c5aeb8&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:51.890770 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830b622-4d35-4601-a5d3-2321c6793b05 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:51.934654 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:51.965435 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68fe3e-ff37-4ef9-bd96-fe1619f5830c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:51.966463 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68fe3e-ff37-4ef9-bd96-fe1619f5830c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:51.966978 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68fe3e-ff37-4ef9-bd96-fe1619f5830c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:51.967454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68fe3e-ff37-4ef9-bd96-fe1619f5830c None admin] PATH_INFO: `/v3/services/network` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:51.968258 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68fe3e-ff37-4ef9-bd96-fe1619f5830c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:51.969840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68fe3e-ff37-4ef9-bd96-fe1619f5830c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-fa68fe3e-ff37-4ef9-bd96-fe1619f5830c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:51.992170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:51.993241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:51.994097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:51.994097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:51.994097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:51.994097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:51.994097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:51.994097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:51.994097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:51.994097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:51.994097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:51.994097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:51.994097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:39:51.994097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:39:51.994097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 14:39:51.994097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:51.996871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 377/775] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:51 2022] GET /identity/v3/services/network => generated 88 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:52.007290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3830b622-4d35-4601-a5d3-2321c6793b05 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:52.013792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68fe3e-ff37-4ef9-bd96-fe1619f5830c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:52.038601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:52.049727 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffba15b-f026-416d-a974-21a2f2da3419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:52.050567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffba15b-f026-416d-a974-21a2f2da3419 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:52.050989 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffba15b-f026-416d-a974-21a2f2da3419 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:52.052645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffba15b-f026-416d-a974-21a2f2da3419 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:52.053422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffba15b-f026-416d-a974-21a2f2da3419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:52.054937 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffba15b-f026-416d-a974-21a2f2da3419 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:52.073863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 399/776] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Sep 19 14:39:51 2022] GET /identity/v3/services => generated 772 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:52.204188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68fe3e-ff37-4ef9-bd96-fe1619f5830c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:52.252752 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee6e7c-c1b0-4cb0-9234-60ee100c6719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:52.253655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee6e7c-c1b0-4cb0-9234-60ee100c6719 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:52.254096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee6e7c-c1b0-4cb0-9234-60ee100c6719 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:52.254649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee6e7c-c1b0-4cb0-9234-60ee100c6719 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:52.256018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee6e7c-c1b0-4cb0-9234-60ee100c6719 None admin] RBAC: Adding query filter params (name=network) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:52.256623 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee6e7c-c1b0-4cb0-9234-60ee100c6719 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:52.258304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee6e7c-c1b0-4cb0-9234-60ee100c6719 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:52.268410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 378/777] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Sep 19 14:39:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:52.274422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffba15b-f026-416d-a974-21a2f2da3419 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:52.283349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815e180-5fd3-43de-9217-720912c8285c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:52.283760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815e180-5fd3-43de-9217-720912c8285c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:52.284082 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815e180-5fd3-43de-9217-720912c8285c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:52.285839 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 379/778] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:52.294876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0035928-3388-4b8e-b910-d06dffb62289 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:52.294876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0035928-3388-4b8e-b910-d06dffb62289 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:52.294876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0035928-3388-4b8e-b910-d06dffb62289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:52.324085 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b0035928-3388-4b8e-b910-d06dffb62289 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:52.325948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b0035928-3388-4b8e-b910-d06dffb62289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:52.407606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffba15b-f026-416d-a974-21a2f2da3419 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:52.410604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:52.437948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 380/779] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:39:52.465709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca32aed-87ce-49c1-9bf4-96d3d28a6890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:52.466854 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca32aed-87ce-49c1-9bf4-96d3d28a6890 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:52.467352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca32aed-87ce-49c1-9bf4-96d3d28a6890 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:52.467631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca32aed-87ce-49c1-9bf4-96d3d28a6890 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:52.468652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca32aed-87ce-49c1-9bf4-96d3d28a6890 None admin] RBAC: Adding query filter params (type=network) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:52.469185 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca32aed-87ce-49c1-9bf4-96d3d28a6890 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:52.470773 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca32aed-87ce-49c1-9bf4-96d3d28a6890 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:52.477157 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 400/780] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Sep 19 14:39:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:52.482636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0035928-3388-4b8e-b910-d06dffb62289 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:52.634466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0035928-3388-4b8e-b910-d06dffb62289 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:52.701237 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240f76d-9a33-4b3e-ac16-9283c6195f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:52.701872 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240f76d-9a33-4b3e-ac16-9283c6195f39 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:52.702161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240f76d-9a33-4b3e-ac16-9283c6195f39 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:52.702439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240f76d-9a33-4b3e-ac16-9283c6195f39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:52.703216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240f76d-9a33-4b3e-ac16-9283c6195f39 None admin] RBAC: Adding query filter params (interface=public, service_id=4390647391384db78bde7a38803e88ba, region_id=RegionOne) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:52.703529 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240f76d-9a33-4b3e-ac16-9283c6195f39 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:52.704937 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240f76d-9a33-4b3e-ac16-9283c6195f39 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:52.717300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 381/781] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Sep 19 14:39:52 2022] GET /identity/v3/endpoints?service_id=4390647391384db78bde7a38803e88ba&interface=public®ion_id=RegionOne => generated 194 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:39:52.722403 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca32aed-87ce-49c1-9bf4-96d3d28a6890 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:52.744873 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240f76d-9a33-4b3e-ac16-9283c6195f39 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:52.806080 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:52.849418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:52.905072 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240f76d-9a33-4b3e-ac16-9283c6195f39 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:52.917750 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca32aed-87ce-49c1-9bf4-96d3d28a6890 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:52.944417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf070cb-6d1f-4a27-ab58-c8d2475eae9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:52.945085 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf070cb-6d1f-4a27-ab58-c8d2475eae9b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:52.945326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf070cb-6d1f-4a27-ab58-c8d2475eae9b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:52.945720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf070cb-6d1f-4a27-ab58-c8d2475eae9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:52.957268 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf070cb-6d1f-4a27-ab58-c8d2475eae9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:52.960156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ab069-38bd-478d-b0a9-bcea7f8b60ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:52.960957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ab069-38bd-478d-b0a9-bcea7f8b60ec None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:52.961316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ab069-38bd-478d-b0a9-bcea7f8b60ec None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:52.961651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ab069-38bd-478d-b0a9-bcea7f8b60ec None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:52.962253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ab069-38bd-478d-b0a9-bcea7f8b60ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-bbf070cb-6d1f-4a27-ab58-c8d2475eae9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:52.962743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:52.963655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:52.965175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:52.965175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:52.965175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:52.965175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:52.965175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:52.965175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:52.965175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:52.965175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:52.965175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:52.965175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:52.965175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:52.965175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:52.965894 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ab069-38bd-478d-b0a9-bcea7f8b60ec None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:52.965974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 382/782] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:39:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:39:52.965974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c84a2-df92-4082-8bec-9e559dd8d25f None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:52.965974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c84a2-df92-4082-8bec-9e559dd8d25f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:52.966155 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c84a2-df92-4082-8bec-9e559dd8d25f None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:52.967598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 383/783] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:52.968825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e423e-86ea-4aa4-9b9f-2aded45f94b2 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:52.969016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e423e-86ea-4aa4-9b9f-2aded45f94b2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:52.969163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e423e-86ea-4aa4-9b9f-2aded45f94b2 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:52.974574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 384/784] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:52.974574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e423e-86ea-4aa4-9b9f-2aded45f94b2 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:52.977025 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 401/785] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Sep 19 14:39:52 2022] GET /identity/v3/services => generated 772 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:52.977269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc740c-1d5b-402e-b142-e36d299d3a6b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:52.977564 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc740c-1d5b-402e-b142-e36d299d3a6b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:52.977843 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc740c-1d5b-402e-b142-e36d299d3a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:53.000337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-06bc740c-1d5b-402e-b142-e36d299d3a6b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:53.001598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-06bc740c-1d5b-402e-b142-e36d299d3a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:53.097335 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 402/786] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:39:53.099011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c8b9d-799e-424c-aa32-65b3bc960a03 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:53.099486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c8b9d-799e-424c-aa32-65b3bc960a03 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:53.099997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c8b9d-799e-424c-aa32-65b3bc960a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:53.129761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-cd6c8b9d-799e-424c-aa32-65b3bc960a03 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:53.132874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cd6c8b9d-799e-424c-aa32-65b3bc960a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:53.137381 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e423e-86ea-4aa4-9b9f-2aded45f94b2 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:53.148560 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:53.195794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1862d4f-697f-4791-9de3-b566d31e77f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:53.197156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1862d4f-697f-4791-9de3-b566d31e77f3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:53.198472 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1862d4f-697f-4791-9de3-b566d31e77f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:53.199005 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1862d4f-697f-4791-9de3-b566d31e77f3 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:53.199990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1862d4f-697f-4791-9de3-b566d31e77f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:53.201060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1862d4f-697f-4791-9de3-b566d31e77f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:53.202724 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1862d4f-697f-4791-9de3-b566d31e77f3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:53.212713 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 385/787] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:53.218564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1862d4f-697f-4791-9de3-b566d31e77f3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:53.220389 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:53.243950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 403/788] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:39:53.338632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1862d4f-697f-4791-9de3-b566d31e77f3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:53.383856 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83515afa-1a99-4a5c-b8b2-427c2d278663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:53.384740 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83515afa-1a99-4a5c-b8b2-427c2d278663 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:53.385170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83515afa-1a99-4a5c-b8b2-427c2d278663 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:53.385612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83515afa-1a99-4a5c-b8b2-427c2d278663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:53.413243 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c8b9d-799e-424c-aa32-65b3bc960a03 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:53.521705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83515afa-1a99-4a5c-b8b2-427c2d278663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:53.523405 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83515afa-1a99-4a5c-b8b2-427c2d278663 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-83515afa-1a99-4a5c-b8b2-427c2d278663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:53.539406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:53.540641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:53.541760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:53.541760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:53.541760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:53.543457 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 386/789] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:53.549151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83515afa-1a99-4a5c-b8b2-427c2d278663 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:53.579309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:53.612151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c8b9d-799e-424c-aa32-65b3bc960a03 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:53.649849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfdfd2-533d-4818-af89-e0bb8a8be8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:53.650735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfdfd2-533d-4818-af89-e0bb8a8be8b6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:53.651162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfdfd2-533d-4818-af89-e0bb8a8be8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:53.651577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfdfd2-533d-4818-af89-e0bb8a8be8b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:53.663372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfdfd2-533d-4818-af89-e0bb8a8be8b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-15bfdfd2-533d-4818-af89-e0bb8a8be8b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:53.668641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:53.669870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:53.670946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:53.670946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:53.670946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:53.670946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:53.670946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:53.670946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:53.670946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:53.670946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:53.670946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:53.670946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:53.670946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:39:53.670946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:53.673303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 404/790] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:39:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:39:53.674558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfdfd2-533d-4818-af89-e0bb8a8be8b6 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:53.680671 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83515afa-1a99-4a5c-b8b2-427c2d278663 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:53.723409 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068b1cf-2473-461d-8f8e-5f0a0f9824ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:53.724336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068b1cf-2473-461d-8f8e-5f0a0f9824ee None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:53.724780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068b1cf-2473-461d-8f8e-5f0a0f9824ee None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:53.725204 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068b1cf-2473-461d-8f8e-5f0a0f9824ee None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:53.726038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068b1cf-2473-461d-8f8e-5f0a0f9824ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:53.726535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068b1cf-2473-461d-8f8e-5f0a0f9824ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:53.726717 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:53.728705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068b1cf-2473-461d-8f8e-5f0a0f9824ee None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:53.744074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 387/791] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:39:53.745238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068b1cf-2473-461d-8f8e-5f0a0f9824ee None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:53.792092 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfdfd2-533d-4818-af89-e0bb8a8be8b6 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:53.833233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764a0e6-2dec-4a2f-a962-2b98a6f005bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:53.833864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764a0e6-2dec-4a2f-a962-2b98a6f005bd None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:53.834169 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764a0e6-2dec-4a2f-a962-2b98a6f005bd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:53.834572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764a0e6-2dec-4a2f-a962-2b98a6f005bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:53.840201 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:53.847288 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764a0e6-2dec-4a2f-a962-2b98a6f005bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-8764a0e6-2dec-4a2f-a962-2b98a6f005bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:39:53.852138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:53.853083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:53.853944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:53.853944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:53.853944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:53.853944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:53.853944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:39:53.853944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:39:53.853944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:39:53.853944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:39:53.853944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:39:53.853944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:39:53.853944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:39:53.853944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:53.855723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068b1cf-2473-461d-8f8e-5f0a0f9824ee None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:53.858754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 405/792] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:39:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:53.859771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764a0e6-2dec-4a2f-a962-2b98a6f005bd None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:53.905814 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ea440-fb2e-4717-b019-1ab161fe861c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:53.906031 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ea440-fb2e-4717-b019-1ab161fe861c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:53.906252 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ea440-fb2e-4717-b019-1ab161fe861c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:53.906507 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ea440-fb2e-4717-b019-1ab161fe861c None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:53.907175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ea440-fb2e-4717-b019-1ab161fe861c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:53.907522 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ea440-fb2e-4717-b019-1ab161fe861c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:53.909039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ea440-fb2e-4717-b019-1ab161fe861c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:53.918220 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 388/793] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:39:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:53.919620 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ea440-fb2e-4717-b019-1ab161fe861c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:53.986657 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764a0e6-2dec-4a2f-a962-2b98a6f005bd None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:53.988935 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:54.026163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ea440-fb2e-4717-b019-1ab161fe861c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:54.030247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b39656-ad35-4b2f-9ee7-3ec8f775bb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:54.031308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b39656-ad35-4b2f-9ee7-3ec8f775bb7a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:54.031839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b39656-ad35-4b2f-9ee7-3ec8f775bb7a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:54.032316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b39656-ad35-4b2f-9ee7-3ec8f775bb7a None admin] PATH_INFO: `/v3/users/nova` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:54.040675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b39656-ad35-4b2f-9ee7-3ec8f775bb7a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:54.042766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b39656-ad35-4b2f-9ee7-3ec8f775bb7a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-f8b39656-ad35-4b2f-9ee7-3ec8f775bb7a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:54.057139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:54.058121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:54.059122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:54.059122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:54.059122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:54.059122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:54.059122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:54.059122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:54.059122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:54.059122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:54.059122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:39:54.059122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:54.065817 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 406/794] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Sep 19 14:39:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:54.065817 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b39656-ad35-4b2f-9ee7-3ec8f775bb7a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:54.070056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dc0e8-bfd1-4f29-858c-48a95506bf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:54.073446 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dc0e8-bfd1-4f29-858c-48a95506bf2e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:54.073446 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dc0e8-bfd1-4f29-858c-48a95506bf2e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:54.073446 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dc0e8-bfd1-4f29-858c-48a95506bf2e None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:54.073446 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dc0e8-bfd1-4f29-858c-48a95506bf2e None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:54.073446 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dc0e8-bfd1-4f29-858c-48a95506bf2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:54.073808 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dc0e8-bfd1-4f29-858c-48a95506bf2e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:54.085985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 389/795] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:39:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:54.085985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dc0e8-bfd1-4f29-858c-48a95506bf2e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:54.090114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:54.196067 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dc0e8-bfd1-4f29-858c-48a95506bf2e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:54.199602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b39656-ad35-4b2f-9ee7-3ec8f775bb7a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:54.233438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329201e5-575e-473a-b0a3-f1a2406912d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:54.234256 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329201e5-575e-473a-b0a3-f1a2406912d4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:54.234630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329201e5-575e-473a-b0a3-f1a2406912d4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:54.235004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329201e5-575e-473a-b0a3-f1a2406912d4 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:54.235858 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329201e5-575e-473a-b0a3-f1a2406912d4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:54.236264 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329201e5-575e-473a-b0a3-f1a2406912d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:54.237848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329201e5-575e-473a-b0a3-f1a2406912d4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:54.244404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8df5e-48bf-4c47-9546-1f717e8b96e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:54.245224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8df5e-48bf-4c47-9546-1f717e8b96e8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:54.245670 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8df5e-48bf-4c47-9546-1f717e8b96e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:54.246092 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8df5e-48bf-4c47-9546-1f717e8b96e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:54.261495 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 390/796] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Sep 19 14:39:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:54.261495 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329201e5-575e-473a-b0a3-f1a2406912d4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:54.264590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8df5e-48bf-4c47-9546-1f717e8b96e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:54.266368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8df5e-48bf-4c47-9546-1f717e8b96e8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-8fa8df5e-48bf-4c47-9546-1f717e8b96e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:54.283448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:54.284470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:54.285406 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:54.285406 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:54.285406 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:54.288396 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 407/797] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:54.289594 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8df5e-48bf-4c47-9546-1f717e8b96e8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:54.307939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:54.328015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:54.371485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329201e5-575e-473a-b0a3-f1a2406912d4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:54.410296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087c1f1-bcab-4682-a308-0f40bce324c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:54.410922 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087c1f1-bcab-4682-a308-0f40bce324c0 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:54.411247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087c1f1-bcab-4682-a308-0f40bce324c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:54.411489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087c1f1-bcab-4682-a308-0f40bce324c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:54.422836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8df5e-48bf-4c47-9546-1f717e8b96e8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:54.423430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087c1f1-bcab-4682-a308-0f40bce324c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:54.425266 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087c1f1-bcab-4682-a308-0f40bce324c0 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-5087c1f1-bcab-4682-a308-0f40bce324c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:54.441778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:54.442792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:54.443709 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:54.443709 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:54.443709 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:54.443709 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 391/798] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:54.444479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087c1f1-bcab-4682-a308-0f40bce324c0 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:54.467361 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1c181-6434-49c7-82e2-4a9cc6b37595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:54.468351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1c181-6434-49c7-82e2-4a9cc6b37595 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:54.469471 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1c181-6434-49c7-82e2-4a9cc6b37595 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:54.469956 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1c181-6434-49c7-82e2-4a9cc6b37595 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:54.548629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1c181-6434-49c7-82e2-4a9cc6b37595 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:54.550520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1c181-6434-49c7-82e2-4a9cc6b37595 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:54.567997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:54.570544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087c1f1-bcab-4682-a308-0f40bce324c0 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:54.588338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:54.615177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead5efc-15c8-4e7a-9e58-d2ddacef8c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:54.615795 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead5efc-15c8-4e7a-9e58-d2ddacef8c22 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:54.616264 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead5efc-15c8-4e7a-9e58-d2ddacef8c22 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:54.616644 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead5efc-15c8-4e7a-9e58-d2ddacef8c22 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:54.617290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead5efc-15c8-4e7a-9e58-d2ddacef8c22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:54.617594 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead5efc-15c8-4e7a-9e58-d2ddacef8c22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:54.619477 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead5efc-15c8-4e7a-9e58-d2ddacef8c22 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:54.635585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 392/799] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:54.636515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead5efc-15c8-4e7a-9e58-d2ddacef8c22 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:54.678978 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 408/800] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:39:54 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:39:54.680296 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1c181-6434-49c7-82e2-4a9cc6b37595 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:54.778309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead5efc-15c8-4e7a-9e58-d2ddacef8c22 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:54.794939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1c181-6434-49c7-82e2-4a9cc6b37595 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:54.810612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:54.815721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ac54f-dbcb-4d76-a259-a3ba9e2ad1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:54.816362 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ac54f-dbcb-4d76-a259-a3ba9e2ad1a7 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:54.816641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ac54f-dbcb-4d76-a259-a3ba9e2ad1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:54.816912 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ac54f-dbcb-4d76-a259-a3ba9e2ad1a7 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:54.817546 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ac54f-dbcb-4d76-a259-a3ba9e2ad1a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:54.817852 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ac54f-dbcb-4d76-a259-a3ba9e2ad1a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:54.819274 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ac54f-dbcb-4d76-a259-a3ba9e2ad1a7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:54.835677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 393/801] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:54.835677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ac54f-dbcb-4d76-a259-a3ba9e2ad1a7 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:54.838719 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ef3a9-b510-4d75-9106-314c10985cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:54.840219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ef3a9-b510-4d75-9106-314c10985cc9 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:54.840513 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ef3a9-b510-4d75-9106-314c10985cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:54.840786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ef3a9-b510-4d75-9106-314c10985cc9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:54.850052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ef3a9-b510-4d75-9106-314c10985cc9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:54.851497 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ef3a9-b510-4d75-9106-314c10985cc9 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-075ef3a9-b510-4d75-9106-314c10985cc9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:54.865029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:54.866055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:54.867005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:54.867005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:54.867005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:54.867005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:54.867005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:54.867005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:54.867005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:54.867005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:54.867005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 14:39:54.867005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:54.867005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 409/802] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:39:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:54.867613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbaec0b-ea65-485c-9bdd-e9aa74e9eb3f None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:54.868044 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbaec0b-ea65-485c-9bdd-e9aa74e9eb3f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:54.868044 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbaec0b-ea65-485c-9bdd-e9aa74e9eb3f None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:54.871864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 410/803] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:54.871864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbaec0b-ea65-485c-9bdd-e9aa74e9eb3f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:54.880210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:54.960532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ac54f-dbcb-4d76-a259-a3ba9e2ad1a7 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:54.998838 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce2c30-c9f7-45cf-9356-42b4f36d9b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:54.999478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce2c30-c9f7-45cf-9356-42b4f36d9b41 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:54.999763 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce2c30-c9f7-45cf-9356-42b4f36d9b41 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:55.000064 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce2c30-c9f7-45cf-9356-42b4f36d9b41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:55.011315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce2c30-c9f7-45cf-9356-42b4f36d9b41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:55.012854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce2c30-c9f7-45cf-9356-42b4f36d9b41 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-54ce2c30-c9f7-45cf-9356-42b4f36d9b41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:55.027775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:55.034089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:55.034999 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:55.034999 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:55.034999 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:55.034999 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 394/804] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:55.034999 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce2c30-c9f7-45cf-9356-42b4f36d9b41 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:55.105216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbaec0b-ea65-485c-9bdd-e9aa74e9eb3f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:55.125382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:55.142295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce2c30-c9f7-45cf-9356-42b4f36d9b41 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:55.143279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c0d529-c65e-4f05-ab92-90f64910c91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:55.143964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0d529-c65e-4f05-ab92-90f64910c91c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:55.144240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0d529-c65e-4f05-ab92-90f64910c91c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:55.144508 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c0d529-c65e-4f05-ab92-90f64910c91c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:55.151737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c0d529-c65e-4f05-ab92-90f64910c91c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:55.153194 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c0d529-c65e-4f05-ab92-90f64910c91c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-94c0d529-c65e-4f05-ab92-90f64910c91c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:39:55.164863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:55.165864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:55.166811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:55.166811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:55.166811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:39:55.166811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:39:55.166811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:39:55.166811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:39:55.166811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:39:55.166811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:39:55.166811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 14:39:55.166811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:55.166811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 411/805] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Sep 19 14:39:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:55.167402 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33ad01-5a7d-4d07-b49c-c7632eb63aaf None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:55.168058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33ad01-5a7d-4d07-b49c-c7632eb63aaf None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:55.168058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33ad01-5a7d-4d07-b49c-c7632eb63aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:55.181789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58def3a-69f1-422e-8f0b-4ff622cdeeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:55.182404 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58def3a-69f1-422e-8f0b-4ff622cdeeab None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:55.182686 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58def3a-69f1-422e-8f0b-4ff622cdeeab None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:55.182958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58def3a-69f1-422e-8f0b-4ff622cdeeab None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:55.183681 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58def3a-69f1-422e-8f0b-4ff622cdeeab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:55.184028 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58def3a-69f1-422e-8f0b-4ff622cdeeab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:55.186016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58def3a-69f1-422e-8f0b-4ff622cdeeab None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:55.186131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2e33ad01-5a7d-4d07-b49c-c7632eb63aaf None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:55.187243 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2e33ad01-5a7d-4d07-b49c-c7632eb63aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:55.207926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 395/806] 10.222.0.60 () {56 vars in 1135 bytes} [Mon Sep 19 14:39:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:55.207926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58def3a-69f1-422e-8f0b-4ff622cdeeab None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:55.220057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:55.265318 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 412/807] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:39:55.266416 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33ad01-5a7d-4d07-b49c-c7632eb63aaf None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:55.331064 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58def3a-69f1-422e-8f0b-4ff622cdeeab None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:55.372658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e39e7-a83c-42a0-8e18-81f0e364e241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:55.373372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e39e7-a83c-42a0-8e18-81f0e364e241 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:55.373754 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e39e7-a83c-42a0-8e18-81f0e364e241 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:55.374127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e39e7-a83c-42a0-8e18-81f0e364e241 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:55.374877 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e39e7-a83c-42a0-8e18-81f0e364e241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:55.375281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e39e7-a83c-42a0-8e18-81f0e364e241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:55.376788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509e39e7-a83c-42a0-8e18-81f0e364e241 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:55.393666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 396/808] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:55.393666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e39e7-a83c-42a0-8e18-81f0e364e241 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:55.396003 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:55.414981 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33ad01-5a7d-4d07-b49c-c7632eb63aaf None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:55.454320 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae43ae0-7cc6-447a-92dc-3b2862b72b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:55.454921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae43ae0-7cc6-447a-92dc-3b2862b72b3f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:55.455202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae43ae0-7cc6-447a-92dc-3b2862b72b3f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:55.455473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae43ae0-7cc6-447a-92dc-3b2862b72b3f None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:55.456344 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae43ae0-7cc6-447a-92dc-3b2862b72b3f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:55.456765 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae43ae0-7cc6-447a-92dc-3b2862b72b3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:55.458289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae43ae0-7cc6-447a-92dc-3b2862b72b3f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:55.477502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 413/809] 10.222.0.60 () {56 vars in 1149 bytes} [Mon Sep 19 14:39:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:55.478336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae43ae0-7cc6-447a-92dc-3b2862b72b3f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:55.497890 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e39e7-a83c-42a0-8e18-81f0e364e241 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:55.506844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:55.536515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19a88b-5649-4f82-a9fb-3ddec00df646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:55.537457 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19a88b-5649-4f82-a9fb-3ddec00df646 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:55.537953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19a88b-5649-4f82-a9fb-3ddec00df646 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:55.538818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19a88b-5649-4f82-a9fb-3ddec00df646 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:55.550444 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19a88b-5649-4f82-a9fb-3ddec00df646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:55.552160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19a88b-5649-4f82-a9fb-3ddec00df646 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-3e19a88b-5649-4f82-a9fb-3ddec00df646 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:55.567406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:55.568684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:55.569612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:55.569612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:55.569612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:55.569612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 397/810] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:55.569848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83a09-6316-4606-8b69-764f35c4d321 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:55.570126 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83a09-6316-4606-8b69-764f35c4d321 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:55.570400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a83a09-6316-4606-8b69-764f35c4d321 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:55.575764 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 398/811] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:55.575764 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a83a09-6316-4606-8b69-764f35c4d321 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:55.598675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae43ae0-7cc6-447a-92dc-3b2862b72b3f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:55.630430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:55.633474 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2297cfd-a56b-4ebd-9efc-89f011794075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:55.634103 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2297cfd-a56b-4ebd-9efc-89f011794075 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:55.634377 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2297cfd-a56b-4ebd-9efc-89f011794075 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:55.634641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2297cfd-a56b-4ebd-9efc-89f011794075 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:55.641241 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2297cfd-a56b-4ebd-9efc-89f011794075 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:55.642664 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2297cfd-a56b-4ebd-9efc-89f011794075 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:55.654164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-a2297cfd-a56b-4ebd-9efc-89f011794075 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:55.654164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:55.654164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:55.654164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:55.654164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:55.654164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:55.654164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:55.654164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:55.654164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:55.654164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:55.654164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:55.654164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:55.654164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:55.654164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:55.654164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:55.655073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:55.656012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:55.656012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:55.656012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:55.656012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:55.656012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:55.656012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:55.656012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:55.656012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:55.656012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 414/812] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:55.656012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2297cfd-a56b-4ebd-9efc-89f011794075 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:55.680123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a83a09-6316-4606-8b69-764f35c4d321 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:55.712364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7cf65-ba06-4abf-9721-e6a2c96580bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:55.712983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7cf65-ba06-4abf-9721-e6a2c96580bd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:55.713262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7cf65-ba06-4abf-9721-e6a2c96580bd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:55.713540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7cf65-ba06-4abf-9721-e6a2c96580bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:55.724121 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7cf65-ba06-4abf-9721-e6a2c96580bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:55.725593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7cf65-ba06-4abf-9721-e6a2c96580bd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:55.738801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-19c7cf65-ba06-4abf-9721-e6a2c96580bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:39:55.740364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:55.741498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:39:55.742407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:39:55.742407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:39:55.742407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:55.742407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 399/813] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:39:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:55.742729 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37ae00-fd41-43bf-8828-00a7b53f0795 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:55.743010 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37ae00-fd41-43bf-8828-00a7b53f0795 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:55.743325 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37ae00-fd41-43bf-8828-00a7b53f0795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:55.760617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ba37ae00-fd41-43bf-8828-00a7b53f0795 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:55.761813 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ba37ae00-fd41-43bf-8828-00a7b53f0795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:55.761933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2297cfd-a56b-4ebd-9efc-89f011794075 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:55.798736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da4a2c-c234-49fc-935e-a548766e3edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:55.799593 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da4a2c-c234-49fc-935e-a548766e3edc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:55.799902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da4a2c-c234-49fc-935e-a548766e3edc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:55.800175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da4a2c-c234-49fc-935e-a548766e3edc None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:55.800807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da4a2c-c234-49fc-935e-a548766e3edc None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:55.801173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da4a2c-c234-49fc-935e-a548766e3edc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:55.802468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da4a2c-c234-49fc-935e-a548766e3edc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:55.814341 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 415/814] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:55.815152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da4a2c-c234-49fc-935e-a548766e3edc None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:55.828627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 400/815] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:39:55.829460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37ae00-fd41-43bf-8828-00a7b53f0795 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:55.873691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:55.941456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da4a2c-c234-49fc-935e-a548766e3edc None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:55.941804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37ae00-fd41-43bf-8828-00a7b53f0795 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:55.977703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610cdd4-5d65-42e0-95ab-b5d147a60752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:55.978317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610cdd4-5d65-42e0-95ab-b5d147a60752 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:55.978593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610cdd4-5d65-42e0-95ab-b5d147a60752 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:55.978866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b610cdd4-5d65-42e0-95ab-b5d147a60752 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:55.979546 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610cdd4-5d65-42e0-95ab-b5d147a60752 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:55.979866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610cdd4-5d65-42e0-95ab-b5d147a60752 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:55.981301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b610cdd4-5d65-42e0-95ab-b5d147a60752 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:55.984601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02136f4c-744f-4714-813e-ea1dff96692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:55.985409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02136f4c-744f-4714-813e-ea1dff96692e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:55.985854 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02136f4c-744f-4714-813e-ea1dff96692e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:55.986274 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02136f4c-744f-4714-813e-ea1dff96692e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:55.987012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02136f4c-744f-4714-813e-ea1dff96692e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:55.988826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02136f4c-744f-4714-813e-ea1dff96692e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:55.993724 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 401/816] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:55.994854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610cdd4-5d65-42e0-95ab-b5d147a60752 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-02136f4c-744f-4714-813e-ea1dff96692e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:55.998338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:55.999546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:56.000596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:56.000596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:56.000596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:56.000596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:56.000596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:56.000596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:56.000596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:56.000596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:56.000596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:56.000596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:56.000596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:39:56.000596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:39:56.000596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 19 14:39:56.000596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:56.004100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 416/817] 10.222.0.60 () {56 vars in 1129 bytes} [Mon Sep 19 14:39:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:56.004941 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02136f4c-744f-4714-813e-ea1dff96692e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:56.014353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:56.108715 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:56.112394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02136f4c-744f-4714-813e-ea1dff96692e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:56.113267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b610cdd4-5d65-42e0-95ab-b5d147a60752 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:56.147847 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273b05f-e232-4c0a-9e5e-84d8f5d1fb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:56.148088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d35eab-718b-47ae-b772-16a486d97fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:56.148478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273b05f-e232-4c0a-9e5e-84d8f5d1fb09 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:56.148648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d35eab-718b-47ae-b772-16a486d97fa2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:56.148762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273b05f-e232-4c0a-9e5e-84d8f5d1fb09 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:56.148934 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d35eab-718b-47ae-b772-16a486d97fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:56.149037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273b05f-e232-4c0a-9e5e-84d8f5d1fb09 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:56.149205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d35eab-718b-47ae-b772-16a486d97fa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:56.149657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273b05f-e232-4c0a-9e5e-84d8f5d1fb09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:56.149960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273b05f-e232-4c0a-9e5e-84d8f5d1fb09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:56.151333 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3273b05f-e232-4c0a-9e5e-84d8f5d1fb09 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:56.157317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d35eab-718b-47ae-b772-16a486d97fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:56.158728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d35eab-718b-47ae-b772-16a486d97fa2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:56.163892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 402/818] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:56.164705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273b05f-e232-4c0a-9e5e-84d8f5d1fb09 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:56.170785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-95d35eab-718b-47ae-b772-16a486d97fa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:56.170785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:56.170785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:56.170785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:56.170785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:56.170785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:56.170785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:56.170785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:56.170785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:56.170785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:56.170785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:56.170785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:56.170785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:56.170785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:56.170785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:56.176847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:56.177986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:56.177986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:56.177986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:56.177986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:56.177986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:56.177986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:56.177986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:56.177986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:56.177986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 417/819] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:56.177986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d35eab-718b-47ae-b772-16a486d97fa2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:56.246059 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:56.267493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273b05f-e232-4c0a-9e5e-84d8f5d1fb09 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:56.282552 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d35eab-718b-47ae-b772-16a486d97fa2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:56.301269 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aedb69-d0eb-445d-b033-1931b416276c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:56.301869 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aedb69-d0eb-445d-b033-1931b416276c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:56.302154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aedb69-d0eb-445d-b033-1931b416276c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:56.302423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aedb69-d0eb-445d-b033-1931b416276c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:56.303140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aedb69-d0eb-445d-b033-1931b416276c None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=59b5d2fd362b417c9b9be75f88a667ad) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:56.303446 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aedb69-d0eb-445d-b033-1931b416276c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:56.304913 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aedb69-d0eb-445d-b033-1931b416276c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:56.311187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 403/820] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:39:56 2022] GET /identity/v3/role_assignments?user.id=59b5d2fd362b417c9b9be75f88a667ad&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:56.311997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aedb69-d0eb-445d-b033-1931b416276c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:56.323389 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1704d2-fc42-4de2-9fd5-88354b549fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:56.324190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1704d2-fc42-4de2-9fd5-88354b549fc2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:56.324533 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1704d2-fc42-4de2-9fd5-88354b549fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:56.324878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1704d2-fc42-4de2-9fd5-88354b549fc2 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:56.325572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1704d2-fc42-4de2-9fd5-88354b549fc2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:56.325873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1704d2-fc42-4de2-9fd5-88354b549fc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:56.327287 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1704d2-fc42-4de2-9fd5-88354b549fc2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:56.339543 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 418/821] 10.222.0.60 () {56 vars in 1124 bytes} [Mon Sep 19 14:39:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:56.339543 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1704d2-fc42-4de2-9fd5-88354b549fc2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:56.361278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:56.458821 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aedb69-d0eb-445d-b033-1931b416276c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:56.460060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1704d2-fc42-4de2-9fd5-88354b549fc2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:56.493918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:56.498584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4cb1e0-d566-4ad0-91c2-0f37d33c0464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:56.499173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4cb1e0-d566-4ad0-91c2-0f37d33c0464 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:56.499458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4cb1e0-d566-4ad0-91c2-0f37d33c0464 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:56.499733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4cb1e0-d566-4ad0-91c2-0f37d33c0464 None admin] PATH_INFO: `/v3/services/network` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:56.500697 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4cb1e0-d566-4ad0-91c2-0f37d33c0464 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:56.501790 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4cb1e0-d566-4ad0-91c2-0f37d33c0464 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:56.502198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4b645-f859-4f94-9e17-eb9664bcde62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:56.503555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4b645-f859-4f94-9e17-eb9664bcde62 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:56.503555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4b645-f859-4f94-9e17-eb9664bcde62 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:56.503555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4b645-f859-4f94-9e17-eb9664bcde62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:56.511450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4b645-f859-4f94-9e17-eb9664bcde62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:56.513738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4b645-f859-4f94-9e17-eb9664bcde62 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-9f4cb1e0-d566-4ad0-91c2-0f37d33c0464 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:56.514969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:56.516287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:56.517140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:56.517140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:56.517140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:56.517140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:56.517140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:56.517140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:56.517140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:56.517140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:56.517140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:56.517140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:56.517140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:39:56.517140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:39:56.517140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 19 14:39:56.517140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:39:56.517140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 404/822] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:56 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:56.518000 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4cb1e0-d566-4ad0-91c2-0f37d33c0464 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:56.532060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-37c4b645-f859-4f94-9e17-eb9664bcde62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:56.532060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:56.532060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:56.532060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:56.532060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:56.532060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:56.532060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:56.532060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:56.532060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:56.532060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:56.532060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:56.532060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:56.532060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:39:56.532060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:39:56.532060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:56.533009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:56.533946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:39:56.533946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:39:56.533946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:39:56.533946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:39:56.533946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:39:56.533946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:39:56.533946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:39:56.533946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:56.533946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 419/823] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:39:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:56.533946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4b645-f859-4f94-9e17-eb9664bcde62 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:56.617720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:56.633996 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4cb1e0-d566-4ad0-91c2-0f37d33c0464 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:56.654419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4b645-f859-4f94-9e17-eb9664bcde62 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:56.669244 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fd2a5-05bd-4d6a-a703-5deb0f0fa8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:56.669946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fd2a5-05bd-4d6a-a703-5deb0f0fa8f8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:56.670296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fd2a5-05bd-4d6a-a703-5deb0f0fa8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:56.670635 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fd2a5-05bd-4d6a-a703-5deb0f0fa8f8 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:56.671387 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fd2a5-05bd-4d6a-a703-5deb0f0fa8f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:56.671866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fd2a5-05bd-4d6a-a703-5deb0f0fa8f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:56.673293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fd2a5-05bd-4d6a-a703-5deb0f0fa8f8 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:56.691047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 405/824] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:56.691047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fd2a5-05bd-4d6a-a703-5deb0f0fa8f8 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:56.697127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132db311-e630-483e-89ec-c9378d75dcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:56.698104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132db311-e630-483e-89ec-c9378d75dcd6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:56.698390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132db311-e630-483e-89ec-c9378d75dcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:56.698664 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132db311-e630-483e-89ec-c9378d75dcd6 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:56.699633 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132db311-e630-483e-89ec-c9378d75dcd6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:56.699999 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132db311-e630-483e-89ec-c9378d75dcd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:56.701902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132db311-e630-483e-89ec-c9378d75dcd6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:56.711861 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 420/825] 10.222.0.60 () {56 vars in 1124 bytes} [Mon Sep 19 14:39:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:56.711861 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132db311-e630-483e-89ec-c9378d75dcd6 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:56.772623 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:56.792830 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fd2a5-05bd-4d6a-a703-5deb0f0fa8f8 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:56.828138 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ff69f-4118-41df-b129-b1c723228ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:56.828756 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ff69f-4118-41df-b129-b1c723228ead None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:56.829036 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ff69f-4118-41df-b129-b1c723228ead None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:56.829308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ff69f-4118-41df-b129-b1c723228ead None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:56.829587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132db311-e630-483e-89ec-c9378d75dcd6 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:56.829933 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ff69f-4118-41df-b129-b1c723228ead None admin] RBAC: Adding query filter params (name=network) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:56.830251 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ff69f-4118-41df-b129-b1c723228ead None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:56.831606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ff69f-4118-41df-b129-b1c723228ead None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:56.836226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 406/826] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Sep 19 14:39:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:56.837008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ff69f-4118-41df-b129-b1c723228ead None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:56.849871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:56.873199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c0d1f-93c7-4f74-8471-44838f2035d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:56.874050 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c0d1f-93c7-4f74-8471-44838f2035d8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:56.874522 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c0d1f-93c7-4f74-8471-44838f2035d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:56.874796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c0d1f-93c7-4f74-8471-44838f2035d8 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:56.875490 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c0d1f-93c7-4f74-8471-44838f2035d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:56.875798 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c0d1f-93c7-4f74-8471-44838f2035d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:56.877334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c0d1f-93c7-4f74-8471-44838f2035d8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:56.898705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 421/827] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:39:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:56.898705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c0d1f-93c7-4f74-8471-44838f2035d8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:56.958540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ff69f-4118-41df-b129-b1c723228ead None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:56.994332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be267a4d-07ba-4411-a0f1-4d9f17780ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:56.995137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be267a4d-07ba-4411-a0f1-4d9f17780ef2 None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:56.995393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be267a4d-07ba-4411-a0f1-4d9f17780ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:56.995688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be267a4d-07ba-4411-a0f1-4d9f17780ef2 None admin] PATH_INFO: `/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/bfc14d32f4314c6e93d20a9c5a45c220/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:56.997691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be267a4d-07ba-4411-a0f1-4d9f17780ef2 None admin] RBAC: Authorizing `identity:create_grant(project_id=7dee98edf19d478ca4f746ff85579fba, user_id=bfc14d32f4314c6e93d20a9c5a45c220, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:56.999138 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be267a4d-07ba-4411-a0f1-4d9f17780ef2 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:57.007970 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c0d1f-93c7-4f74-8471-44838f2035d8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:57.013671 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 407/828] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:39:56 2022] PUT /identity/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/bfc14d32f4314c6e93d20a9c5a45c220/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:39:57.014489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be267a4d-07ba-4411-a0f1-4d9f17780ef2 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:57.025079 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:57.056078 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ca03b-d3ed-40c2-8b31-69f677a219ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:57.056954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ca03b-d3ed-40c2-8b31-69f677a219ed None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:57.057486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ca03b-d3ed-40c2-8b31-69f677a219ed None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:57.057902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ca03b-d3ed-40c2-8b31-69f677a219ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:57.058732 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ca03b-d3ed-40c2-8b31-69f677a219ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:57.060435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ca03b-d3ed-40c2-8b31-69f677a219ed None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:57.106904 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 422/829] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:39:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:39:57.106904 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ca03b-d3ed-40c2-8b31-69f677a219ed None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:57.162606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:57.229063 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be267a4d-07ba-4411-a0f1-4d9f17780ef2 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:57.280655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72659367-2ddf-4382-a321-ed3e911040bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:57.281478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72659367-2ddf-4382-a321-ed3e911040bd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:57.281758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72659367-2ddf-4382-a321-ed3e911040bd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:57.282029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72659367-2ddf-4382-a321-ed3e911040bd None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:57.282925 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72659367-2ddf-4382-a321-ed3e911040bd None admin] RBAC: Adding query filter params (type=network) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:57.283227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72659367-2ddf-4382-a321-ed3e911040bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:57.285477 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72659367-2ddf-4382-a321-ed3e911040bd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:57.291185 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ca03b-d3ed-40c2-8b31-69f677a219ed None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:57.296054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 408/830] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Sep 19 14:39:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:57.303313 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72659367-2ddf-4382-a321-ed3e911040bd None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:57.340513 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee323d-a573-420b-86cb-6c884e074fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:57.341296 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee323d-a573-420b-86cb-6c884e074fd3 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:57.341578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee323d-a573-420b-86cb-6c884e074fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:57.341855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee323d-a573-420b-86cb-6c884e074fd3 None admin] PATH_INFO: `/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/0fefb7fa6f484341a19a1a8bee31b116/roles/15fbc4d519f64f4cacecc3b20fe826ae` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:57.362498 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee323d-a573-420b-86cb-6c884e074fd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=7dee98edf19d478ca4f746ff85579fba, user_id=0fefb7fa6f484341a19a1a8bee31b116, role_id=15fbc4d519f64f4cacecc3b20fe826ae)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:57.364733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee323d-a573-420b-86cb-6c884e074fd3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:57.383772 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 423/831] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:39:57 2022] PUT /identity/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/0fefb7fa6f484341a19a1a8bee31b116/roles/15fbc4d519f64f4cacecc3b20fe826ae => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:39:57.439536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72659367-2ddf-4382-a321-ed3e911040bd None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:57.479455 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:57.487541 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade13d8-4e0c-4fef-9c9b-513f014cd472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:57.488289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade13d8-4e0c-4fef-9c9b-513f014cd472 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:57.488572 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade13d8-4e0c-4fef-9c9b-513f014cd472 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:57.488840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade13d8-4e0c-4fef-9c9b-513f014cd472 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:57.489510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade13d8-4e0c-4fef-9c9b-513f014cd472 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:57.490980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade13d8-4e0c-4fef-9c9b-513f014cd472 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:57.521920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 409/832] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:39:57 2022] POST /identity/v3/endpoints => generated 337 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:39:58.623959 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8b557-b1d4-4bb2-9402-f54c33efd8e6 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:58.624488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8b557-b1d4-4bb2-9402-f54c33efd8e6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:58.624488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8b557-b1d4-4bb2-9402-f54c33efd8e6 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:58.626097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 424/833] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:58.631663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3f300-e220-4064-87ab-61a46cf93a19 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:58.632130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3f300-e220-4064-87ab-61a46cf93a19 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:58.632277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3f300-e220-4064-87ab-61a46cf93a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:58.650775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0ed3f300-e220-4064-87ab-61a46cf93a19 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:58.652061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0ed3f300-e220-4064-87ab-61a46cf93a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:58.739967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 410/834] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:39:59.001439 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8b557-b1d4-4bb2-9402-f54c33efd8e6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:59.023710 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:59.106264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee8b557-b1d4-4bb2-9402-f54c33efd8e6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:59.150159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16de1b-199e-4b89-97fa-10ab083c8646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:59.150796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16de1b-199e-4b89-97fa-10ab083c8646 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:59.151074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16de1b-199e-4b89-97fa-10ab083c8646 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:59.151344 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16de1b-199e-4b89-97fa-10ab083c8646 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:59.151965 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16de1b-199e-4b89-97fa-10ab083c8646 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:59.153357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16de1b-199e-4b89-97fa-10ab083c8646 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-6d16de1b-199e-4b89-97fa-10ab083c8646 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:39:59.163182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:39:59.164279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:39:59.165266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:39:59.165266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:39:59.165266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:39:59.165266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:39:59.165266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:39:59.165266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:39:59.165266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:39:59.165266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:39:59.165266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:39:59.165266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:39:59.165266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:39:59.165266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:39:59.165266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 14:39:59.165266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:39:59.165266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 425/835] 10.222.0.60 () {56 vars in 1132 bytes} [Mon Sep 19 14:39:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:39:59.171293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3f300-e220-4064-87ab-61a46cf93a19 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:59.278202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3f300-e220-4064-87ab-61a46cf93a19 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:59.293066 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:59.318372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e49e0-c087-47e3-a262-6d20cb5dc77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:59.318991 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e49e0-c087-47e3-a262-6d20cb5dc77a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:59.319266 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e49e0-c087-47e3-a262-6d20cb5dc77a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:59.319534 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e49e0-c087-47e3-a262-6d20cb5dc77a None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:59.320186 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e49e0-c087-47e3-a262-6d20cb5dc77a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:59.320484 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e49e0-c087-47e3-a262-6d20cb5dc77a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:59.321994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e49e0-c087-47e3-a262-6d20cb5dc77a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:59.327865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 411/836] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Sep 19 14:39:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:59.333321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16de1b-199e-4b89-97fa-10ab083c8646 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:59.429649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:39:59.446917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16de1b-199e-4b89-97fa-10ab083c8646 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:39:59.481987 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b716d-4eaa-4793-96f3-d2a0a9c55cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:39:59.482596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b716d-4eaa-4793-96f3-d2a0a9c55cb4 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:59.482875 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b716d-4eaa-4793-96f3-d2a0a9c55cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:59.483147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b716d-4eaa-4793-96f3-d2a0a9c55cb4 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:59.483770 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b716d-4eaa-4793-96f3-d2a0a9c55cb4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:39:59.484104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b716d-4eaa-4793-96f3-d2a0a9c55cb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:39:59.485466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b716d-4eaa-4793-96f3-d2a0a9c55cb4 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:39:59.490373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 426/837] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Sep 19 14:39:59 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:39:59.491402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdb0d2-15e4-42b6-ad60-dfcc37ced20f None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:59.491689 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdb0d2-15e4-42b6-ad60-dfcc37ced20f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:59.491996 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdb0d2-15e4-42b6-ad60-dfcc37ced20f None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:59.493328 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 412/838] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:59.499720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612dd22-caf5-4165-8fd1-6adaca246b14 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:59.500078 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612dd22-caf5-4165-8fd1-6adaca246b14 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:59.500299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612dd22-caf5-4165-8fd1-6adaca246b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:59.520556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c612dd22-caf5-4165-8fd1-6adaca246b14 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:59.521856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c612dd22-caf5-4165-8fd1-6adaca246b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:59.598765 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 427/839] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:39:59.803866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8d01b-5c9c-40eb-8249-324017020621 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:59.804412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8d01b-5c9c-40eb-8249-324017020621 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:59.804412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8d01b-5c9c-40eb-8249-324017020621 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:59.805994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 413/840] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:39:59.811414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b85f9f-542a-4484-8b75-f0c282ebc118 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:59.811866 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b85f9f-542a-4484-8b75-f0c282ebc118 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:59.812053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b85f9f-542a-4484-8b75-f0c282ebc118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:59.818081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e364-0355-40e6-ad92-282b5b2f3983 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:59.818648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e364-0355-40e6-ad92-282b5b2f3983 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:59.819159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e364-0355-40e6-ad92-282b5b2f3983 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:59.820799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 414/841] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:39:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:39:59.828460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d5690-d333-4931-98ad-4ec4499a02d8 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:39:59.829049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d5690-d333-4931-98ad-4ec4499a02d8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:39:59.829534 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d5690-d333-4931-98ad-4ec4499a02d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:39:59.840725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f0b85f9f-542a-4484-8b75-f0c282ebc118 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:59.842376 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f0b85f9f-542a-4484-8b75-f0c282ebc118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:59.850716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6c5d5690-d333-4931-98ad-4ec4499a02d8 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:39:59.852415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6c5d5690-d333-4931-98ad-4ec4499a02d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:39:59.934547 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 428/842] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:39:59.935108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 415/843] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:39:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:39:59.936718 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b85f9f-542a-4484-8b75-f0c282ebc118 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:39:59.950152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:00.050106 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b85f9f-542a-4484-8b75-f0c282ebc118 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:00.098506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848480bd-b63e-4ecb-ad79-8ab38ba2c06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:00.099448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848480bd-b63e-4ecb-ad79-8ab38ba2c06b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:00.099962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848480bd-b63e-4ecb-ad79-8ab38ba2c06b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:00.100447 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848480bd-b63e-4ecb-ad79-8ab38ba2c06b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:00.112082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848480bd-b63e-4ecb-ad79-8ab38ba2c06b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-848480bd-b63e-4ecb-ad79-8ab38ba2c06b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:40:00.117218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:00.118162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:00.119020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:00.119020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:00.119020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:00.119020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:00.119020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:40:00.119020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:40:00.119020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:40:00.119020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:40:00.119020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:40:00.119020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:40:00.119020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:40:00.119020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:00.121745 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 429/844] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:39:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:00.127336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d5690-d333-4931-98ad-4ec4499a02d8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:00.209767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848480bd-b63e-4ecb-ad79-8ab38ba2c06b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:00.247668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d5690-d333-4931-98ad-4ec4499a02d8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:00.255947 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:00.289016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f4bc0-2d16-4d38-afe5-09239a9ea75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:00.289972 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f4bc0-2d16-4d38-afe5-09239a9ea75c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:00.290445 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f4bc0-2d16-4d38-afe5-09239a9ea75c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:00.290897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f4bc0-2d16-4d38-afe5-09239a9ea75c None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:00.291746 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f4bc0-2d16-4d38-afe5-09239a9ea75c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:00.292082 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f4bc0-2d16-4d38-afe5-09239a9ea75c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:00.292451 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:00.293460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f4bc0-2d16-4d38-afe5-09239a9ea75c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:00.312689 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 416/845] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:40:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:00.312689 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f4bc0-2d16-4d38-afe5-09239a9ea75c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:00.331899 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848480bd-b63e-4ecb-ad79-8ab38ba2c06b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:00.367780 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea939e-971f-475e-9500-4eea1b64fc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:00.368358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea939e-971f-475e-9500-4eea1b64fc31 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:00.368641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea939e-971f-475e-9500-4eea1b64fc31 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:00.368907 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea939e-971f-475e-9500-4eea1b64fc31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:00.379000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea939e-971f-475e-9500-4eea1b64fc31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-53ea939e-971f-475e-9500-4eea1b64fc31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:40:00.383730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:00.384730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:00.389065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:00.389065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:00.389065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:00.389065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:00.389065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:40:00.389065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:40:00.389065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:40:00.389065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:40:00.389065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:40:00.389065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:40:00.389065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:40:00.389065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:00.389065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 430/846] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:40:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:00.389065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea939e-971f-475e-9500-4eea1b64fc31 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:00.427424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f4bc0-2d16-4d38-afe5-09239a9ea75c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:00.463879 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298b3e0-a11d-46c9-918d-03683f4350c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:00.464864 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298b3e0-a11d-46c9-918d-03683f4350c8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:00.465339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298b3e0-a11d-46c9-918d-03683f4350c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:00.466312 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298b3e0-a11d-46c9-918d-03683f4350c8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:00.467332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298b3e0-a11d-46c9-918d-03683f4350c8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:00.469050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298b3e0-a11d-46c9-918d-03683f4350c8 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-f298b3e0-a11d-46c9-918d-03683f4350c8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:00.478708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:00.479742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:00.480653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:00.480653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:00.480653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:00.480653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:00.480653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:00.480653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:00.480653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:00.480653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:00.480653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:00.480653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:00.480653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:00.480653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:00.480653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 14:40:00.480653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:00.481407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 417/847] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:40:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:00.482804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298b3e0-a11d-46c9-918d-03683f4350c8 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:00.495314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea939e-971f-475e-9500-4eea1b64fc31 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:00.518401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:00.524343 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:00.535071 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec06f63-bfd1-4ed6-810f-14b44e626bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:00.535705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec06f63-bfd1-4ed6-810f-14b44e626bfa None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:00.536004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec06f63-bfd1-4ed6-810f-14b44e626bfa None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:00.536273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec06f63-bfd1-4ed6-810f-14b44e626bfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:00.546877 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec06f63-bfd1-4ed6-810f-14b44e626bfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:00.548347 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec06f63-bfd1-4ed6-810f-14b44e626bfa None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-2ec06f63-bfd1-4ed6-810f-14b44e626bfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:00.563949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:00.565148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:00.566245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:00.566245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:00.566245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:00.566245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 431/848] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:00.570438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec06f63-bfd1-4ed6-810f-14b44e626bfa None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:00.601256 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298b3e0-a11d-46c9-918d-03683f4350c8 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:00.639172 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72c849-9bb3-4782-8201-91e6d965c7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:00.639699 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72c849-9bb3-4782-8201-91e6d965c7c9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:00.640000 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72c849-9bb3-4782-8201-91e6d965c7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:00.640271 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e72c849-9bb3-4782-8201-91e6d965c7c9 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:00.640925 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72c849-9bb3-4782-8201-91e6d965c7c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:00.641227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72c849-9bb3-4782-8201-91e6d965c7c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:00.642586 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e72c849-9bb3-4782-8201-91e6d965c7c9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:00.656491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 418/849] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:40:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:00.656491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72c849-9bb3-4782-8201-91e6d965c7c9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:00.695772 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec06f63-bfd1-4ed6-810f-14b44e626bfa None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:00.731479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5559c-8da2-42d6-a9d5-d8541afa279c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:00.732188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5559c-8da2-42d6-a9d5-d8541afa279c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:00.732471 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5559c-8da2-42d6-a9d5-d8541afa279c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:00.732748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5559c-8da2-42d6-a9d5-d8541afa279c None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:00.733370 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5559c-8da2-42d6-a9d5-d8541afa279c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:00.733690 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5559c-8da2-42d6-a9d5-d8541afa279c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:00.735039 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5559c-8da2-42d6-a9d5-d8541afa279c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:00.740283 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 432/850] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:40:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:00.740901 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5559c-8da2-42d6-a9d5-d8541afa279c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:00.765065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:00.773204 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e72c849-9bb3-4782-8201-91e6d965c7c9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:00.792115 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:00.808605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a1c5c-04ec-40d6-859d-f08a3d1f3559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:00.809271 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a1c5c-04ec-40d6-859d-f08a3d1f3559 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:00.809584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a1c5c-04ec-40d6-859d-f08a3d1f3559 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:00.810396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a1c5c-04ec-40d6-859d-f08a3d1f3559 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:00.811037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a1c5c-04ec-40d6-859d-f08a3d1f3559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:00.811340 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a1c5c-04ec-40d6-859d-f08a3d1f3559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:00.812739 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a1c5c-04ec-40d6-859d-f08a3d1f3559 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:00.830614 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 419/851] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:00.830614 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a1c5c-04ec-40d6-859d-f08a3d1f3559 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:00.855805 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5559c-8da2-42d6-a9d5-d8541afa279c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:00.899849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6bcbf-2768-4d86-9d20-aa29312dfab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:00.900517 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6bcbf-2768-4d86-9d20-aa29312dfab1 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:00.900896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6bcbf-2768-4d86-9d20-aa29312dfab1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:00.901073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6bcbf-2768-4d86-9d20-aa29312dfab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:00.913189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6bcbf-2768-4d86-9d20-aa29312dfab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:00.914640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6bcbf-2768-4d86-9d20-aa29312dfab1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-6ef6bcbf-2768-4d86-9d20-aa29312dfab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:00.929321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:00.930541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:00.931751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:00.931751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:00.931751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:00.936143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 433/852] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:00.937037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6bcbf-2768-4d86-9d20-aa29312dfab1 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:00.947767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a1c5c-04ec-40d6-859d-f08a3d1f3559 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:00.987073 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5c513-3a8a-4435-a2f1-d3a9b0d1b163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:00.988285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5c513-3a8a-4435-a2f1-d3a9b0d1b163 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:00.988802 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5c513-3a8a-4435-a2f1-d3a9b0d1b163 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:00.989381 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5c513-3a8a-4435-a2f1-d3a9b0d1b163 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:00.990535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5c513-3a8a-4435-a2f1-d3a9b0d1b163 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:00.991156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5c513-3a8a-4435-a2f1-d3a9b0d1b163 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:00.993780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5c513-3a8a-4435-a2f1-d3a9b0d1b163 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:01.007849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 420/853] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:40:00 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:01.007849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5c513-3a8a-4435-a2f1-d3a9b0d1b163 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:01.027452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:01.057738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:01.070357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6bcbf-2768-4d86-9d20-aa29312dfab1 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:01.117423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef81cb-f9ea-4bde-b5f1-9e3b449b00ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:01.118379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef81cb-f9ea-4bde-b5f1-9e3b449b00ae None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:01.118809 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef81cb-f9ea-4bde-b5f1-9e3b449b00ae None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:01.119259 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef81cb-f9ea-4bde-b5f1-9e3b449b00ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:01.131006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef81cb-f9ea-4bde-b5f1-9e3b449b00ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:01.132872 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef81cb-f9ea-4bde-b5f1-9e3b449b00ae None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-64ef81cb-f9ea-4bde-b5f1-9e3b449b00ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:40:01.145815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:01.147100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:01.148351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:01.148351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:01.148351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:40:01.148351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:40:01.148351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:40:01.148351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:40:01.148351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:40:01.148351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:40:01.148351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 19 14:40:01.148351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:01.150813 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 434/854] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:40:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:01.156833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef81cb-f9ea-4bde-b5f1-9e3b449b00ae None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:01.166339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5c513-3a8a-4435-a2f1-d3a9b0d1b163 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:01.221642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4089681-1464-4b46-ad54-fdde9580c9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:01.222629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4089681-1464-4b46-ad54-fdde9580c9d8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:01.222912 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4089681-1464-4b46-ad54-fdde9580c9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:01.223230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4089681-1464-4b46-ad54-fdde9580c9d8 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:01.223863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4089681-1464-4b46-ad54-fdde9580c9d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:01.224171 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4089681-1464-4b46-ad54-fdde9580c9d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:01.225543 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4089681-1464-4b46-ad54-fdde9580c9d8 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:01.242334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 421/855] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:01.250870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4089681-1464-4b46-ad54-fdde9580c9d8 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:01.279664 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef81cb-f9ea-4bde-b5f1-9e3b449b00ae None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:01.317326 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06aaa7-264f-4cbb-a95e-eaff9be1aa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:01.317948 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06aaa7-264f-4cbb-a95e-eaff9be1aa9c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:01.318233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06aaa7-264f-4cbb-a95e-eaff9be1aa9c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:01.318624 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06aaa7-264f-4cbb-a95e-eaff9be1aa9c None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:01.319369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06aaa7-264f-4cbb-a95e-eaff9be1aa9c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:01.319788 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06aaa7-264f-4cbb-a95e-eaff9be1aa9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:01.321261 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06aaa7-264f-4cbb-a95e-eaff9be1aa9c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:01.333803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:01.344733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 435/856] 10.222.0.60 () {56 vars in 1135 bytes} [Mon Sep 19 14:40:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:01.350909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06aaa7-264f-4cbb-a95e-eaff9be1aa9c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:01.363488 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4089681-1464-4b46-ad54-fdde9580c9d8 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:01.368931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:01.400042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d55948-18f1-4b9f-afa4-64d25c26a0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:01.400729 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d55948-18f1-4b9f-afa4-64d25c26a0ab None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:01.401138 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d55948-18f1-4b9f-afa4-64d25c26a0ab None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:01.401362 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d55948-18f1-4b9f-afa4-64d25c26a0ab None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:01.409554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d55948-18f1-4b9f-afa4-64d25c26a0ab None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:01.411117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d55948-18f1-4b9f-afa4-64d25c26a0ab None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-31d55948-18f1-4b9f-afa4-64d25c26a0ab None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:40:01.423429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:01.424468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:01.425706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:01.425706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:01.425706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:40:01.425706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:40:01.425706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:40:01.425706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:40:01.425706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:40:01.425706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:40:01.425706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 14:40:01.425706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:01.425706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 422/857] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Sep 19 14:40:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:01.430175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d55948-18f1-4b9f-afa4-64d25c26a0ab None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:01.482018 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06aaa7-264f-4cbb-a95e-eaff9be1aa9c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:01.531507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35e785-66a9-43f9-ac23-f005cf381dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:01.532254 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35e785-66a9-43f9-ac23-f005cf381dc2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:01.532616 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35e785-66a9-43f9-ac23-f005cf381dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:01.532963 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35e785-66a9-43f9-ac23-f005cf381dc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:01.563668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d55948-18f1-4b9f-afa4-64d25c26a0ab None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:01.614849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cbcb9-f51b-4e42-8b8f-556e3dc87832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:01.616356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cbcb9-f51b-4e42-8b8f-556e3dc87832 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:01.616828 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cbcb9-f51b-4e42-8b8f-556e3dc87832 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:01.617324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cbcb9-f51b-4e42-8b8f-556e3dc87832 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:01.618233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71cbcb9-f51b-4e42-8b8f-556e3dc87832 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:01.618766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71cbcb9-f51b-4e42-8b8f-556e3dc87832 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:01.621078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71cbcb9-f51b-4e42-8b8f-556e3dc87832 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:01.622933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35e785-66a9-43f9-ac23-f005cf381dc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:01.624744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35e785-66a9-43f9-ac23-f005cf381dc2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:01.632798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:01.645982 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:01.648879 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 423/858] 10.222.0.60 () {56 vars in 1149 bytes} [Mon Sep 19 14:40:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:01.655076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cbcb9-f51b-4e42-8b8f-556e3dc87832 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:01.772116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 436/859] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:40:01 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:40:01.773447 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35e785-66a9-43f9-ac23-f005cf381dc2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:01.786013 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cbcb9-f51b-4e42-8b8f-556e3dc87832 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:01.821892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a54404-3d01-40c6-92f3-88f1226933fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:01.822538 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a54404-3d01-40c6-92f3-88f1226933fb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:01.822992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a54404-3d01-40c6-92f3-88f1226933fb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:01.823235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a54404-3d01-40c6-92f3-88f1226933fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:01.893402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:01.896235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35e785-66a9-43f9-ac23-f005cf381dc2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:01.902309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a54404-3d01-40c6-92f3-88f1226933fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:01.903918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a54404-3d01-40c6-92f3-88f1226933fb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:01.912798 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:01.936690 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef7482-8a64-492f-9fd2-860444e15bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:01.937331 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef7482-8a64-492f-9fd2-860444e15bb8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:01.937613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef7482-8a64-492f-9fd2-860444e15bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:01.937887 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef7482-8a64-492f-9fd2-860444e15bb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:01.948628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef7482-8a64-492f-9fd2-860444e15bb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:01.950074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef7482-8a64-492f-9fd2-860444e15bb8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-7cef7482-8a64-492f-9fd2-860444e15bb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:01.970349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:01.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:01.972352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:01.972352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:01.972352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:01.972352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 437/860] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:01.974528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3360db-ff89-477b-b67b-463aa4e7b890 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:01.974808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3360db-ff89-477b-b67b-463aa4e7b890 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:01.975084 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3360db-ff89-477b-b67b-463aa4e7b890 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:01.979883 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 438/861] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:01.979883 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3360db-ff89-477b-b67b-463aa4e7b890 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:02.023109 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 424/862] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:40:01 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:40:02.024420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08bed4-18a1-4a94-8cb1-62ec436d7e15 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:02.024708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08bed4-18a1-4a94-8cb1-62ec436d7e15 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:02.024985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08bed4-18a1-4a94-8cb1-62ec436d7e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:02.043093 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ac08bed4-18a1-4a94-8cb1-62ec436d7e15 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:02.044850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ac08bed4-18a1-4a94-8cb1-62ec436d7e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:02.107215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3360db-ff89-477b-b67b-463aa4e7b890 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:02.124281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 425/863] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:02.125444 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08bed4-18a1-4a94-8cb1-62ec436d7e15 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:02.156163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:02.159873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bac69d-40e3-4653-b891-ccdb9385e4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:02.161411 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bac69d-40e3-4653-b891-ccdb9385e4ff None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:02.161910 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bac69d-40e3-4653-b891-ccdb9385e4ff None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:02.162782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bac69d-40e3-4653-b891-ccdb9385e4ff None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:02.163940 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bac69d-40e3-4653-b891-ccdb9385e4ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:02.164417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bac69d-40e3-4653-b891-ccdb9385e4ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:02.166205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bac69d-40e3-4653-b891-ccdb9385e4ff None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:02.184055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 439/864] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:02.189586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bac69d-40e3-4653-b891-ccdb9385e4ff None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:02.226954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:02.244360 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08bed4-18a1-4a94-8cb1-62ec436d7e15 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:02.285292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07775dd7-90cf-4485-b715-01b14a15d7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:02.286253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07775dd7-90cf-4485-b715-01b14a15d7a9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:02.286736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07775dd7-90cf-4485-b715-01b14a15d7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:02.287211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07775dd7-90cf-4485-b715-01b14a15d7a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:02.298581 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07775dd7-90cf-4485-b715-01b14a15d7a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:02.300651 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07775dd7-90cf-4485-b715-01b14a15d7a9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-07775dd7-90cf-4485-b715-01b14a15d7a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:02.317166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:02.318164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:02.319080 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:02.319080 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:02.319080 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:02.321245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 426/865] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:02.325925 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bac69d-40e3-4653-b891-ccdb9385e4ff None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:02.327051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07775dd7-90cf-4485-b715-01b14a15d7a9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:02.366486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee89813-5604-4a5e-9ed7-bc0619673dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:02.367320 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee89813-5604-4a5e-9ed7-bc0619673dcb None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:02.367600 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee89813-5604-4a5e-9ed7-bc0619673dcb None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:02.368158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee89813-5604-4a5e-9ed7-bc0619673dcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:02.376505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee89813-5604-4a5e-9ed7-bc0619673dcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:02.378392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee89813-5604-4a5e-9ed7-bc0619673dcb None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:02.390761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-7ee89813-5604-4a5e-9ed7-bc0619673dcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:02.390761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:02.390761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:02.390761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:02.390761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:02.390761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:02.390761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:02.390761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:02.390761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:02.390761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:02.390761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:02.390761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:02.390761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:40:02.390761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:40:02.390761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:02.392006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:02.393147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:40:02.393147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:40:02.393147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:40:02.393147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:40:02.393147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:40:02.393147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:40:02.393147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:02.393147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:02.395212 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 440/866] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:40:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:02.400246 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee89813-5604-4a5e-9ed7-bc0619673dcb None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:02.437997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:02.463322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07775dd7-90cf-4485-b715-01b14a15d7a9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:02.496778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:02.499336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb35ea3-8b44-4b32-acf2-72d1ec9583b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:02.499994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb35ea3-8b44-4b32-acf2-72d1ec9583b2 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:02.500278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb35ea3-8b44-4b32-acf2-72d1ec9583b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:02.500556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb35ea3-8b44-4b32-acf2-72d1ec9583b2 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:02.501196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb35ea3-8b44-4b32-acf2-72d1ec9583b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:02.501500 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb35ea3-8b44-4b32-acf2-72d1ec9583b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:02.502884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb35ea3-8b44-4b32-acf2-72d1ec9583b2 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:02.514178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee89813-5604-4a5e-9ed7-bc0619673dcb None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:02.519863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 427/867] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:02.519863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb35ea3-8b44-4b32-acf2-72d1ec9583b2 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:02.555055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38785f-5824-43e7-9766-cd871baa976c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:02.555657 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38785f-5824-43e7-9766-cd871baa976c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:02.555979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38785f-5824-43e7-9766-cd871baa976c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:02.556915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38785f-5824-43e7-9766-cd871baa976c None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:02.557042 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38785f-5824-43e7-9766-cd871baa976c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:02.557258 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38785f-5824-43e7-9766-cd871baa976c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:02.558665 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38785f-5824-43e7-9766-cd871baa976c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:02.572504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 441/868] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:40:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:02.573144 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38785f-5824-43e7-9766-cd871baa976c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:02.631115 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb35ea3-8b44-4b32-acf2-72d1ec9583b2 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:02.676948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae92c1-4e75-4908-bb41-efa62d848a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:02.677838 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae92c1-4e75-4908-bb41-efa62d848a33 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:02.678275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae92c1-4e75-4908-bb41-efa62d848a33 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:02.678704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae92c1-4e75-4908-bb41-efa62d848a33 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:02.679506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae92c1-4e75-4908-bb41-efa62d848a33 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:02.681233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae92c1-4e75-4908-bb41-efa62d848a33 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:02.692961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:02.694948 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38785f-5824-43e7-9766-cd871baa976c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:02.708031 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 428/869] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Sep 19 14:40:02 2022] POST /identity/v3/services => generated 269 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:40:02.709672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae92c1-4e75-4908-bb41-efa62d848a33 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:02.757721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c9538-d8dd-4811-bef1-7f6b924fc076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:02.759130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c9538-d8dd-4811-bef1-7f6b924fc076 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:02.759538 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c9538-d8dd-4811-bef1-7f6b924fc076 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:02.759945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c9538-d8dd-4811-bef1-7f6b924fc076 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:02.768973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c9538-d8dd-4811-bef1-7f6b924fc076 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:02.770641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c9538-d8dd-4811-bef1-7f6b924fc076 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:02.785613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-d60c9538-d8dd-4811-bef1-7f6b924fc076 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:02.785613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:02.785613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:02.785613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:02.785613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:02.785613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:02.785613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:02.785613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:02.785613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:02.785613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:02.785613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:02.785613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:02.785613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:40:02.785613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:40:02.785613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:02.786799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:02.787815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:40:02.787815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:40:02.787815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:40:02.787815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:40:02.787815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:40:02.787815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:40:02.787815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:02.787815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:02.793502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 442/870] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:40:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:02.795627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c9538-d8dd-4811-bef1-7f6b924fc076 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:02.799222 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:02.847906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae92c1-4e75-4908-bb41-efa62d848a33 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:02.896337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3340c-9a68-4877-92ed-adb7ae75f139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:02.896563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3340c-9a68-4877-92ed-adb7ae75f139 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:02.897940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3340c-9a68-4877-92ed-adb7ae75f139 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:02.897940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd3340c-9a68-4877-92ed-adb7ae75f139 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:02.897940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3340c-9a68-4877-92ed-adb7ae75f139 None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=bfc14d32f4314c6e93d20a9c5a45c220) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:02.898197 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3340c-9a68-4877-92ed-adb7ae75f139 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:02.899565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd3340c-9a68-4877-92ed-adb7ae75f139 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:02.909102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 429/871] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:40:02 2022] GET /identity/v3/role_assignments?user.id=bfc14d32f4314c6e93d20a9c5a45c220&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:02.934334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c9538-d8dd-4811-bef1-7f6b924fc076 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:02.978331 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147323d6-d030-4e45-b843-3ea323b4f80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:02.978885 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147323d6-d030-4e45-b843-3ea323b4f80b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:02.979165 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147323d6-d030-4e45-b843-3ea323b4f80b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:02.979434 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147323d6-d030-4e45-b843-3ea323b4f80b None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:02.980186 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147323d6-d030-4e45-b843-3ea323b4f80b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:02.980492 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147323d6-d030-4e45-b843-3ea323b4f80b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:02.982482 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147323d6-d030-4e45-b843-3ea323b4f80b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:02.997254 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 443/872] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:40:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:03.003415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3340c-9a68-4877-92ed-adb7ae75f139 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:03.076563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:03.125281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd3340c-9a68-4877-92ed-adb7ae75f139 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:03.166066 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca8626-3f24-4308-97e8-97f9700113f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:03.166704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca8626-3f24-4308-97e8-97f9700113f8 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:03.166987 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca8626-3f24-4308-97e8-97f9700113f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:03.167260 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca8626-3f24-4308-97e8-97f9700113f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:03.170440 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca8626-3f24-4308-97e8-97f9700113f8 None admin] RBAC: Adding query filter params (role.id=15fbc4d519f64f4cacecc3b20fe826ae, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=0fefb7fa6f484341a19a1a8bee31b116) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:03.170440 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca8626-3f24-4308-97e8-97f9700113f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:03.170661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca8626-3f24-4308-97e8-97f9700113f8 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:03.178123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 430/873] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:40:03 2022] GET /identity/v3/role_assignments?user.id=0fefb7fa6f484341a19a1a8bee31b116&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=15fbc4d519f64f4cacecc3b20fe826ae => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:03.277061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d15684-db98-40b4-8dff-7f7d4217a6f0 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:03.277706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d15684-db98-40b4-8dff-7f7d4217a6f0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:03.278186 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d15684-db98-40b4-8dff-7f7d4217a6f0 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:03.280624 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 444/874] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:03.286709 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf581331-46e0-4ccb-b0d1-c7ec83155747 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:03.287355 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf581331-46e0-4ccb-b0d1-c7ec83155747 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:03.287901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf581331-46e0-4ccb-b0d1-c7ec83155747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:03.308460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-cf581331-46e0-4ccb-b0d1-c7ec83155747 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:03.310149 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cf581331-46e0-4ccb-b0d1-c7ec83155747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:03.385742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 431/875] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:03 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:03.654415 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d15684-db98-40b4-8dff-7f7d4217a6f0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:03.720960 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:03.766610 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d15684-db98-40b4-8dff-7f7d4217a6f0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:03.802557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45682295-743a-4699-9bc7-59799b564e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:03.803170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45682295-743a-4699-9bc7-59799b564e06 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:03.803451 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45682295-743a-4699-9bc7-59799b564e06 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:03.803724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45682295-743a-4699-9bc7-59799b564e06 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:03.804423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45682295-743a-4699-9bc7-59799b564e06 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:03.805780 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45682295-743a-4699-9bc7-59799b564e06 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:03.822130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 445/876] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Sep 19 14:40:03 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:40:05.069975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c99d97-2c4d-4b3d-b263-6ae990db9e0f None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:05.070565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c99d97-2c4d-4b3d-b263-6ae990db9e0f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:05.073772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c99d97-2c4d-4b3d-b263-6ae990db9e0f None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:05.078488 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 432/877] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:05.089946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaecd98-5a69-4f62-bf04-f623a8e4f33d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:05.090736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaecd98-5a69-4f62-bf04-f623a8e4f33d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:05.091266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaecd98-5a69-4f62-bf04-f623a8e4f33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:05.110585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ccaecd98-5a69-4f62-bf04-f623a8e4f33d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:05.112110 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ccaecd98-5a69-4f62-bf04-f623a8e4f33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:05.173904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30923f-0832-420c-a523-7a0d8b33dfeb None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:05.174296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30923f-0832-420c-a523-7a0d8b33dfeb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:05.174467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30923f-0832-420c-a523-7a0d8b33dfeb None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:05.177375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 433/878] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:40:05.181589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30a1ec-0651-4680-894e-0586e98bf765 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:05.181876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30a1ec-0651-4680-894e-0586e98bf765 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:05.182156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30a1ec-0651-4680-894e-0586e98bf765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:05.192601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 446/879] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:05.200835 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ea30a1ec-0651-4680-894e-0586e98bf765 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:05.202321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ea30a1ec-0651-4680-894e-0586e98bf765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:05.253083 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:05.280016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 434/880] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:40:05.470556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaecd98-5a69-4f62-bf04-f623a8e4f33d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:05.479508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0eaad-a147-499d-a2eb-661d71190706 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:05.479799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0eaad-a147-499d-a2eb-661d71190706 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:05.480107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0eaad-a147-499d-a2eb-661d71190706 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:05.481844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 435/881] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:40:05.486910 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ed7a6-d369-423d-b016-34fd388ce26d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:05.487200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ed7a6-d369-423d-b016-34fd388ce26d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:05.487476 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667ed7a6-d369-423d-b016-34fd388ce26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:05.506195 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-667ed7a6-d369-423d-b016-34fd388ce26d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:05.507393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-667ed7a6-d369-423d-b016-34fd388ce26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:05.521661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:05.578797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaecd98-5a69-4f62-bf04-f623a8e4f33d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:05.592563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 436/882] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:05.593488 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ed7a6-d369-423d-b016-34fd388ce26d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:05.630353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43ce12-e1b6-45c8-9075-d010dbc0a1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:05.631014 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43ce12-e1b6-45c8-9075-d010dbc0a1db None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:05.631259 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43ce12-e1b6-45c8-9075-d010dbc0a1db None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:05.631528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43ce12-e1b6-45c8-9075-d010dbc0a1db None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:05.632204 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43ce12-e1b6-45c8-9075-d010dbc0a1db None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:05.633851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43ce12-e1b6-45c8-9075-d010dbc0a1db None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-be43ce12-e1b6-45c8-9075-d010dbc0a1db None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:05.644628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:05.645661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:05.646581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:05.646581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:05.646581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:05.646581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:05.646581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:05.646581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:05.646581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:05.646581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:05.646581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:05.646581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:05.646581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:05.646581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:05.646581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 14:40:05.646581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:05.650570 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 447/883] 10.222.0.60 () {56 vars in 1132 bytes} [Mon Sep 19 14:40:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:05.651571 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43ce12-e1b6-45c8-9075-d010dbc0a1db None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:05.713194 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667ed7a6-d369-423d-b016-34fd388ce26d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:05.731441 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:05.751735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0dba3-95d4-4a12-a77f-72570c892ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:05.752737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0dba3-95d4-4a12-a77f-72570c892ee9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:05.753275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0dba3-95d4-4a12-a77f-72570c892ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:05.753790 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0dba3-95d4-4a12-a77f-72570c892ee9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:05.754601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0dba3-95d4-4a12-a77f-72570c892ee9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:05.756311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0dba3-95d4-4a12-a77f-72570c892ee9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-8ea0dba3-95d4-4a12-a77f-72570c892ee9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:05.766145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:05.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:05.768451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:05.768451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:05.768451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:05.768451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:05.768451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:05.768451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:05.768451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:05.768451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:05.768451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:05.768451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:05.768451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:05.768451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:05.768451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 14:40:05.768451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:05.771009 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 437/884] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Sep 19 14:40:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:05.775868 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43ce12-e1b6-45c8-9075-d010dbc0a1db None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:05.776529 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0dba3-95d4-4a12-a77f-72570c892ee9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:05.789815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:05.814537 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544916b5-bc1a-4a33-8bc0-13f4df6a79b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:05.815142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544916b5-bc1a-4a33-8bc0-13f4df6a79b6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:05.815504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544916b5-bc1a-4a33-8bc0-13f4df6a79b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:05.815694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544916b5-bc1a-4a33-8bc0-13f4df6a79b6 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:05.816345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544916b5-bc1a-4a33-8bc0-13f4df6a79b6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:05.816641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544916b5-bc1a-4a33-8bc0-13f4df6a79b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:05.818326 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544916b5-bc1a-4a33-8bc0-13f4df6a79b6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:05.827593 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 448/885] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Sep 19 14:40:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:05.830684 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544916b5-bc1a-4a33-8bc0-13f4df6a79b6 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:05.917663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0dba3-95d4-4a12-a77f-72570c892ee9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:05.953193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8fc036-e08b-4d8d-bb69-af2cbff5ecdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:05.953825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fc036-e08b-4d8d-bb69-af2cbff5ecdd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:05.954103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fc036-e08b-4d8d-bb69-af2cbff5ecdd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:05.954369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8fc036-e08b-4d8d-bb69-af2cbff5ecdd None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:05.954777 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544916b5-bc1a-4a33-8bc0-13f4df6a79b6 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:05.955009 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8fc036-e08b-4d8d-bb69-af2cbff5ecdd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:05.955370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8fc036-e08b-4d8d-bb69-af2cbff5ecdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:05.956824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8fc036-e08b-4d8d-bb69-af2cbff5ecdd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:05.967155 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 438/886] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Sep 19 14:40:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:05.967155 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8fc036-e08b-4d8d-bb69-af2cbff5ecdd None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:06.004990 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d65dcb-2173-4d44-82f0-cccb9a03af84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:06.005818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d65dcb-2173-4d44-82f0-cccb9a03af84 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:06.005818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d65dcb-2173-4d44-82f0-cccb9a03af84 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:06.006116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d65dcb-2173-4d44-82f0-cccb9a03af84 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:06.006696 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d65dcb-2173-4d44-82f0-cccb9a03af84 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:06.006997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d65dcb-2173-4d44-82f0-cccb9a03af84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:06.008417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d65dcb-2173-4d44-82f0-cccb9a03af84 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:06.019520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 449/887] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Sep 19 14:40:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:06.019520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d65dcb-2173-4d44-82f0-cccb9a03af84 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:06.019813 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:06.067002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:06.083424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8fc036-e08b-4d8d-bb69-af2cbff5ecdd None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:06.118970 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524d0f3-a2eb-4e64-bd6f-7cf147a68791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:06.119627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524d0f3-a2eb-4e64-bd6f-7cf147a68791 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:06.119945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524d0f3-a2eb-4e64-bd6f-7cf147a68791 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:06.120224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524d0f3-a2eb-4e64-bd6f-7cf147a68791 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:06.130931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524d0f3-a2eb-4e64-bd6f-7cf147a68791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:40:06.131237 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d65dcb-2173-4d44-82f0-cccb9a03af84 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-5524d0f3-a2eb-4e64-bd6f-7cf147a68791 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:40:06.136192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:06.137165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:06.138139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:06.138139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:06.138139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:06.138139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:06.138139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:40:06.138139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:40:06.138139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:40:06.138139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:40:06.138139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:40:06.138139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:40:06.138139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 14:40:06.138139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:06.138139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 439/888] 10.222.0.60 () {56 vars in 1120 bytes} [Mon Sep 19 14:40:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:06.138139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524d0f3-a2eb-4e64-bd6f-7cf147a68791 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:06.167185 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d3f65-a000-4a51-abf6-82e443270e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:06.167790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d3f65-a000-4a51-abf6-82e443270e8a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:06.168116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d3f65-a000-4a51-abf6-82e443270e8a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:06.168387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d3f65-a000-4a51-abf6-82e443270e8a None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:06.169098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d3f65-a000-4a51-abf6-82e443270e8a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:06.169398 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d3f65-a000-4a51-abf6-82e443270e8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:06.170749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d3f65-a000-4a51-abf6-82e443270e8a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:06.175401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 450/889] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Sep 19 14:40:06 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:06.176583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a9f77-483c-4abc-892c-5242c1c767ad None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:06.176863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a9f77-483c-4abc-892c-5242c1c767ad None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:06.177136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a9f77-483c-4abc-892c-5242c1c767ad None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:06.178504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 451/890] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:06.180695 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a9f77-483c-4abc-892c-5242c1c767ad None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:06.253992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524d0f3-a2eb-4e64-bd6f-7cf147a68791 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:06.256788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:06.293564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d4ada-8079-4586-ac7c-abec678aba54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:06.294332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d4ada-8079-4586-ac7c-abec678aba54 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:06.294609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d4ada-8079-4586-ac7c-abec678aba54 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:06.294998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d4ada-8079-4586-ac7c-abec678aba54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:06.295712 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d4ada-8079-4586-ac7c-abec678aba54 None admin] RBAC: Adding query filter params (interface=public, service_id=a8414f2345264697bf6e1e3fcb96a831, region_id=RegionOne) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:06.296052 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d4ada-8079-4586-ac7c-abec678aba54 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:06.297462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d4ada-8079-4586-ac7c-abec678aba54 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:06.303416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 440/891] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Sep 19 14:40:06 2022] GET /identity/v3/endpoints?service_id=a8414f2345264697bf6e1e3fcb96a831&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:06.304731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcdd8b-538d-44f0-ad2a-3f14cd3139e9 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:06.305019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcdd8b-538d-44f0-ad2a-3f14cd3139e9 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:06.305293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcdd8b-538d-44f0-ad2a-3f14cd3139e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:06.310717 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a9f77-483c-4abc-892c-5242c1c767ad None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:06.313957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:06.327247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ddfcdd8b-538d-44f0-ad2a-3f14cd3139e9 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:06.328511 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ddfcdd8b-538d-44f0-ad2a-3f14cd3139e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:06.353732 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2dadc8-e193-4c4e-b275-2c73bf3c3b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:06.354359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dadc8-e193-4c4e-b275-2c73bf3c3b8f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:06.354643 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dadc8-e193-4c4e-b275-2c73bf3c3b8f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:06.354925 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2dadc8-e193-4c4e-b275-2c73bf3c3b8f None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:06.356033 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2dadc8-e193-4c4e-b275-2c73bf3c3b8f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:06.356033 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2dadc8-e193-4c4e-b275-2c73bf3c3b8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:06.357354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2dadc8-e193-4c4e-b275-2c73bf3c3b8f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:06.366801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 452/892] 10.222.0.60 () {56 vars in 1115 bytes} [Mon Sep 19 14:40:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:06.367863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2dadc8-e193-4c4e-b275-2c73bf3c3b8f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:06.409811 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 441/893] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:06.410681 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcdd8b-538d-44f0-ad2a-3f14cd3139e9 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:06.508112 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2dadc8-e193-4c4e-b275-2c73bf3c3b8f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:06.520233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:06.524408 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcdd8b-538d-44f0-ad2a-3f14cd3139e9 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:06.559400 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8385a-492a-4a2b-afb8-367e17731b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:06.561203 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8385a-492a-4a2b-afb8-367e17731b4d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:06.562061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8385a-492a-4a2b-afb8-367e17731b4d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:06.562776 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8385a-492a-4a2b-afb8-367e17731b4d None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:06.563572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8385a-492a-4a2b-afb8-367e17731b4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:06.565137 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8385a-492a-4a2b-afb8-367e17731b4d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:06.566689 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebbedc-403a-4954-94e6-051e5f327459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:06.567467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebbedc-403a-4954-94e6-051e5f327459 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:06.567751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebbedc-403a-4954-94e6-051e5f327459 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:06.568233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceebbedc-403a-4954-94e6-051e5f327459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:06.573447 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 453/894] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Sep 19 14:40:06 2022] GET /identity/v3/services => generated 1268 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:40:06.579590 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebbedc-403a-4954-94e6-051e5f327459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:06.581117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceebbedc-403a-4954-94e6-051e5f327459 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-ceebbedc-403a-4954-94e6-051e5f327459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:06.603222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:06.604302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:06.605215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:06.605215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:06.605215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:06.605215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 442/895] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:06.610937 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8385a-492a-4a2b-afb8-367e17731b4d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:06.646923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:06.712185 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebbedc-403a-4954-94e6-051e5f327459 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:06.729687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8385a-492a-4a2b-afb8-367e17731b4d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:06.772617 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bacd3d2-f749-45a6-a276-0ebc61ba408c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:06.773317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacd3d2-f749-45a6-a276-0ebc61ba408c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:06.773660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacd3d2-f749-45a6-a276-0ebc61ba408c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:06.774251 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacd3d2-f749-45a6-a276-0ebc61ba408c None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:06.774894 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bacd3d2-f749-45a6-a276-0ebc61ba408c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:06.775194 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bacd3d2-f749-45a6-a276-0ebc61ba408c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:06.776799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bacd3d2-f749-45a6-a276-0ebc61ba408c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:06.795402 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 454/896] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:06.802322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bacd3d2-f749-45a6-a276-0ebc61ba408c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:06.832423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceebbedc-403a-4954-94e6-051e5f327459 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:06.871947 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab551a-c94b-4a96-a03e-75ce461ad6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:06.873001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab551a-c94b-4a96-a03e-75ce461ad6bd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:06.873573 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab551a-c94b-4a96-a03e-75ce461ad6bd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:06.874143 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fab551a-c94b-4a96-a03e-75ce461ad6bd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:06.874988 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab551a-c94b-4a96-a03e-75ce461ad6bd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:06.876772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fab551a-c94b-4a96-a03e-75ce461ad6bd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-3fab551a-c94b-4a96-a03e-75ce461ad6bd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:06.887183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:06.888588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:06.889647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:06.889647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:06.889647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:06.889647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:06.889647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:06.889647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:06.889647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:06.889647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:06.889647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:06.889647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:06.889647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:06.889647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:06.889647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 14:40:06.889647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:06.892577 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 443/897] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:40:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:06.898470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab551a-c94b-4a96-a03e-75ce461ad6bd None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:06.917862 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:06.927260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:06.940150 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bacd3d2-f749-45a6-a276-0ebc61ba408c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:06.981004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ec87c-507f-4cf7-b87a-864dcd8bff6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:06.981876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ec87c-507f-4cf7-b87a-864dcd8bff6d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:06.982829 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ec87c-507f-4cf7-b87a-864dcd8bff6d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:06.983210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ec87c-507f-4cf7-b87a-864dcd8bff6d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:06.994468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ec87c-507f-4cf7-b87a-864dcd8bff6d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:06.996950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ec87c-507f-4cf7-b87a-864dcd8bff6d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-1a7ec87c-507f-4cf7-b87a-864dcd8bff6d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:40:07.014269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:07.015560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:07.016778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:07.016778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:07.016778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:40:07.016778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:40:07.016778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:40:07.016778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:40:07.016778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:40:07.016778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:40:07.016778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 14:40:07.016778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:07.023775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 455/898] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Sep 19 14:40:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:07.025045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ec87c-507f-4cf7-b87a-864dcd8bff6d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:07.026782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fab551a-c94b-4a96-a03e-75ce461ad6bd None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:07.064073 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfa0f9-61bf-4e35-adca-5a092c4d4607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:07.064983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfa0f9-61bf-4e35-adca-5a092c4d4607 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:07.065426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfa0f9-61bf-4e35-adca-5a092c4d4607 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:07.065844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfa0f9-61bf-4e35-adca-5a092c4d4607 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:07.066690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfa0f9-61bf-4e35-adca-5a092c4d4607 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:07.067158 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfa0f9-61bf-4e35-adca-5a092c4d4607 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:07.068756 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfa0f9-61bf-4e35-adca-5a092c4d4607 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:07.077572 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 444/899] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:40:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:07.084449 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfa0f9-61bf-4e35-adca-5a092c4d4607 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:07.165825 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ec87c-507f-4cf7-b87a-864dcd8bff6d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:07.202257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:07.203718 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f4c19-5f74-4c6e-82ec-80148070d481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:07.204354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f4c19-5f74-4c6e-82ec-80148070d481 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:07.204628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f4c19-5f74-4c6e-82ec-80148070d481 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:07.204894 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f4c19-5f74-4c6e-82ec-80148070d481 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:07.205807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f4c19-5f74-4c6e-82ec-80148070d481 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:07.206144 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f4c19-5f74-4c6e-82ec-80148070d481 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:07.207753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f4c19-5f74-4c6e-82ec-80148070d481 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:07.226684 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 456/900] 10.222.0.60 () {56 vars in 1149 bytes} [Mon Sep 19 14:40:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:07.231997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f4c19-5f74-4c6e-82ec-80148070d481 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:07.235230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfa0f9-61bf-4e35-adca-5a092c4d4607 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:07.243369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:07.278857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9f5aa-4869-4d6f-806f-6d7278137d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:07.279569 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9f5aa-4869-4d6f-806f-6d7278137d1e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:07.279973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9f5aa-4869-4d6f-806f-6d7278137d1e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:07.280362 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9f5aa-4869-4d6f-806f-6d7278137d1e None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:07.281083 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9f5aa-4869-4d6f-806f-6d7278137d1e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:07.281473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9f5aa-4869-4d6f-806f-6d7278137d1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:07.283108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9f5aa-4869-4d6f-806f-6d7278137d1e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:07.293175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 445/901] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:40:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:07.295731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9f5aa-4869-4d6f-806f-6d7278137d1e None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:07.351836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f4c19-5f74-4c6e-82ec-80148070d481 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:07.391229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e1b82-05cf-47f6-b296-563f95a74487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:07.392785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e1b82-05cf-47f6-b296-563f95a74487 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:07.392785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e1b82-05cf-47f6-b296-563f95a74487 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:07.392785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e1b82-05cf-47f6-b296-563f95a74487 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:07.432609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9f5aa-4869-4d6f-806f-6d7278137d1e None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:07.475672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e1b82-05cf-47f6-b296-563f95a74487 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:07.477782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e1b82-05cf-47f6-b296-563f95a74487 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:07.478521 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b18c9-5eda-4635-b634-fa05acd0df40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:07.479639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b18c9-5eda-4635-b634-fa05acd0df40 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:07.480072 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b18c9-5eda-4635-b634-fa05acd0df40 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:07.480656 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b18c9-5eda-4635-b634-fa05acd0df40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:07.482440 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b18c9-5eda-4635-b634-fa05acd0df40 None admin] RBAC: Adding query filter params (interface=public, service_id=a3972511dfd14b74b425befa0f4560a1, region_id=RegionOne) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:07.482874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b18c9-5eda-4635-b634-fa05acd0df40 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:07.484443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b18c9-5eda-4635-b634-fa05acd0df40 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:07.491857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 446/902] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Sep 19 14:40:07 2022] GET /identity/v3/endpoints?service_id=a3972511dfd14b74b425befa0f4560a1&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:07.496592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b18c9-5eda-4635-b634-fa05acd0df40 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:07.496945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:07.512558 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:07.612067 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 457/903] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:40:07 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 19 14:40:07.614574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e1b82-05cf-47f6-b296-563f95a74487 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:07.637790 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b18c9-5eda-4635-b634-fa05acd0df40 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:07.676106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad8748-9d9b-48b5-b073-102919e988e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:07.677104 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad8748-9d9b-48b5-b073-102919e988e6 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:07.677612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad8748-9d9b-48b5-b073-102919e988e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:07.678102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad8748-9d9b-48b5-b073-102919e988e6 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:07.678894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad8748-9d9b-48b5-b073-102919e988e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:07.680506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad8748-9d9b-48b5-b073-102919e988e6 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:07.692341 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 447/904] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Sep 19 14:40:07 2022] GET /identity/v3/services => generated 1268 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:40:07.733393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e1b82-05cf-47f6-b296-563f95a74487 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:07.744367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:07.775140 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4fafa-0256-4c64-a848-7723fdd89951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:07.775794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4fafa-0256-4c64-a848-7723fdd89951 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:07.776282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4fafa-0256-4c64-a848-7723fdd89951 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:07.776782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4fafa-0256-4c64-a848-7723fdd89951 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:07.792252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4fafa-0256-4c64-a848-7723fdd89951 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:07.793754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4fafa-0256-4c64-a848-7723fdd89951 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-37b4fafa-0256-4c64-a848-7723fdd89951 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:07.811818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:07.812879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:07.813785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:07.813785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:07.813785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:07.813785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 458/905] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:07.820235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad8748-9d9b-48b5-b073-102919e988e6 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:07.905013 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:07.939579 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad8748-9d9b-48b5-b073-102919e988e6 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:07.977615 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59107731-cb78-4f99-9e43-3e5889ae33f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:07.978226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59107731-cb78-4f99-9e43-3e5889ae33f6 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:07.978507 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59107731-cb78-4f99-9e43-3e5889ae33f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:07.978777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59107731-cb78-4f99-9e43-3e5889ae33f6 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:07.979422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59107731-cb78-4f99-9e43-3e5889ae33f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:07.979720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59107731-cb78-4f99-9e43-3e5889ae33f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:07.981280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59107731-cb78-4f99-9e43-3e5889ae33f6 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:07.994478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 448/906] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:08.000548 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4fafa-0256-4c64-a848-7723fdd89951 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:08.108424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4fafa-0256-4c64-a848-7723fdd89951 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:08.145591 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a51a0-0085-4f80-9ec7-8abdaf4c44ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:08.146287 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a51a0-0085-4f80-9ec7-8abdaf4c44ba None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:08.146571 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a51a0-0085-4f80-9ec7-8abdaf4c44ba None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:08.146843 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a51a0-0085-4f80-9ec7-8abdaf4c44ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:08.154238 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a51a0-0085-4f80-9ec7-8abdaf4c44ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:08.155744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a51a0-0085-4f80-9ec7-8abdaf4c44ba None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:08.167368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-e38a51a0-0085-4f80-9ec7-8abdaf4c44ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:08.167368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:08.167368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:08.167368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:08.167368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:08.167368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:08.167368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:08.167368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:08.167368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:08.167368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:08.167368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:08.167368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:08.167368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:40:08.167368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:40:08.167368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:08.168927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:08.170560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:40:08.170560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:40:08.170560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:40:08.170560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:40:08.170560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:40:08.170560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:40:08.170560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:08.170560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:08.170560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 459/907] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:40:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:08.175088 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59107731-cb78-4f99-9e43-3e5889ae33f6 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:08.282640 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59107731-cb78-4f99-9e43-3e5889ae33f6 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:08.322940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:08.331139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813324b-ae0f-4180-9e75-dd043093e361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:08.331758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813324b-ae0f-4180-9e75-dd043093e361 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:08.332067 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813324b-ae0f-4180-9e75-dd043093e361 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:08.332339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813324b-ae0f-4180-9e75-dd043093e361 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:08.333033 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813324b-ae0f-4180-9e75-dd043093e361 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:08.333339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813324b-ae0f-4180-9e75-dd043093e361 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:08.334846 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813324b-ae0f-4180-9e75-dd043093e361 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:08.348836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 449/908] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:40:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:08.357289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a51a0-0085-4f80-9ec7-8abdaf4c44ba None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:08.381574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:08.411322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea0af5-baa2-40da-aec2-3fd10e72a469 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:08.411507 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea0af5-baa2-40da-aec2-3fd10e72a469 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:08.411789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea0af5-baa2-40da-aec2-3fd10e72a469 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:08.413361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 450/909] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:40:08.418254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a01ff2-ad23-4c42-afd7-ce9f1118500b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:08.418539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a01ff2-ad23-4c42-afd7-ce9f1118500b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:08.418814 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a01ff2-ad23-4c42-afd7-ce9f1118500b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:08.437755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a2a01ff2-ad23-4c42-afd7-ce9f1118500b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:08.438952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a2a01ff2-ad23-4c42-afd7-ce9f1118500b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:08.474593 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a51a0-0085-4f80-9ec7-8abdaf4c44ba None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:08.520366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb66eea-0736-43c4-ba9f-e40e5cdffeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:08.520981 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb66eea-0736-43c4-ba9f-e40e5cdffeae None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:08.521259 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb66eea-0736-43c4-ba9f-e40e5cdffeae None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:08.521531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb66eea-0736-43c4-ba9f-e40e5cdffeae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:08.522259 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb66eea-0736-43c4-ba9f-e40e5cdffeae None admin] RBAC: Adding query filter params (role.id=b88a5181c4564a8ca1acebcbd45465c0, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=0fefb7fa6f484341a19a1a8bee31b116) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:08.522567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb66eea-0736-43c4-ba9f-e40e5cdffeae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:08.524094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb66eea-0736-43c4-ba9f-e40e5cdffeae None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:08.525075 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 451/910] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:08 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:40:08.530428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 460/911] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:40:08 2022] GET /identity/v3/role_assignments?user.id=0fefb7fa6f484341a19a1a8bee31b116&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=b88a5181c4564a8ca1acebcbd45465c0 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:08.797474 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a01ff2-ad23-4c42-afd7-ce9f1118500b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:08.904988 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a01ff2-ad23-4c42-afd7-ce9f1118500b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:08.927728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fa749-bc92-4468-aa78-226aefd4e358 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:08.928211 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fa749-bc92-4468-aa78-226aefd4e358 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:08.928296 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94fa749-bc92-4468-aa78-226aefd4e358 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:08.929795 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 461/912] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:08.935117 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33570d91-ed25-4235-8a62-c58cda5f9e65 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:08.935260 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:08.935749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33570d91-ed25-4235-8a62-c58cda5f9e65 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:08.936290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33570d91-ed25-4235-8a62-c58cda5f9e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:08.952119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d3ddd-7cf5-46c1-a7c4-9094a7d661b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:08.952496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d3ddd-7cf5-46c1-a7c4-9094a7d661b0 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:08.952720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d3ddd-7cf5-46c1-a7c4-9094a7d661b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:08.952998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d3ddd-7cf5-46c1-a7c4-9094a7d661b0 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:08.953651 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d3ddd-7cf5-46c1-a7c4-9094a7d661b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:08.955032 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3d3ddd-7cf5-46c1-a7c4-9094a7d661b0 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:08.958967 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-33570d91-ed25-4235-8a62-c58cda5f9e65 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:08.960630 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-33570d91-ed25-4235-8a62-c58cda5f9e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:08.972061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 452/913] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Sep 19 14:40:08 2022] POST /identity/v3/services => generated 241 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:40:09.057236 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 462/914] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:40:09.333609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d3ddd-7cf5-46c1-a7c4-9094a7d661b0 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:09.442765 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d3ddd-7cf5-46c1-a7c4-9094a7d661b0 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:09.478289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf072d-04f0-47ff-9d44-5aab84f4598a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:09.478924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf072d-04f0-47ff-9d44-5aab84f4598a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:09.479200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf072d-04f0-47ff-9d44-5aab84f4598a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:09.479471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf072d-04f0-47ff-9d44-5aab84f4598a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:09.480077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf072d-04f0-47ff-9d44-5aab84f4598a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:09.481547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf072d-04f0-47ff-9d44-5aab84f4598a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-40cf072d-04f0-47ff-9d44-5aab84f4598a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:09.490722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:09.491676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:09.492648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:09.492648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:09.492648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:09.492648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:09.492648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:09.492648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:09.492648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:09.492648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:09.492648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:09.492648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:09.492648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:09.492648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:09.492648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 19 14:40:09.492648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:09.492648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 453/915] 10.222.0.60 () {56 vars in 1132 bytes} [Mon Sep 19 14:40:09 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:09.498186 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33570d91-ed25-4235-8a62-c58cda5f9e65 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:09.509837 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:09.598995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33570d91-ed25-4235-8a62-c58cda5f9e65 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:09.634150 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523cef6-0644-4008-a856-ef68537a8270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:09.634754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523cef6-0644-4008-a856-ef68537a8270 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:09.635029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523cef6-0644-4008-a856-ef68537a8270 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:09.635295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523cef6-0644-4008-a856-ef68537a8270 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:09.635990 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523cef6-0644-4008-a856-ef68537a8270 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:09.636292 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523cef6-0644-4008-a856-ef68537a8270 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:09.638211 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523cef6-0644-4008-a856-ef68537a8270 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:09.643296 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 463/916] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Sep 19 14:40:09 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:09.648941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf072d-04f0-47ff-9d44-5aab84f4598a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:09.698439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:09.758245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf072d-04f0-47ff-9d44-5aab84f4598a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:09.796403 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e424626-08e8-40bf-afd5-bf5e8aab4997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:09.797418 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e424626-08e8-40bf-afd5-bf5e8aab4997 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:09.797925 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e424626-08e8-40bf-afd5-bf5e8aab4997 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:09.798406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e424626-08e8-40bf-afd5-bf5e8aab4997 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:09.799240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e424626-08e8-40bf-afd5-bf5e8aab4997 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:09.800023 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e424626-08e8-40bf-afd5-bf5e8aab4997 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:09.801614 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e424626-08e8-40bf-afd5-bf5e8aab4997 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:09.807559 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 454/917] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Sep 19 14:40:09 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:09.813179 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523cef6-0644-4008-a856-ef68537a8270 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:09.894583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:09.925270 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523cef6-0644-4008-a856-ef68537a8270 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:09.963524 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eed50f-23ce-48be-9535-a57a91c1ab6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:09.964162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eed50f-23ce-48be-9535-a57a91c1ab6d None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:09.964440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eed50f-23ce-48be-9535-a57a91c1ab6d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:09.964713 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eed50f-23ce-48be-9535-a57a91c1ab6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:09.965386 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eed50f-23ce-48be-9535-a57a91c1ab6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:09.967447 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eed50f-23ce-48be-9535-a57a91c1ab6d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:09.988674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 464/918] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:40:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:40:10.017456 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a9754-587c-4b78-88a7-33f5c1a92e45 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:10.017780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a9754-587c-4b78-88a7-33f5c1a92e45 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:10.018016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a9754-587c-4b78-88a7-33f5c1a92e45 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:10.019777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 455/919] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:10.025675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1f412-8c95-404d-91d5-c177f0bd1009 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:10.026071 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1f412-8c95-404d-91d5-c177f0bd1009 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:10.026350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1f412-8c95-404d-91d5-c177f0bd1009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:10.047283 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b6c1f412-8c95-404d-91d5-c177f0bd1009 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:10.048642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b6c1f412-8c95-404d-91d5-c177f0bd1009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:10.141215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 465/920] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:10.412163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a9754-587c-4b78-88a7-33f5c1a92e45 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:10.521835 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a9754-587c-4b78-88a7-33f5c1a92e45 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:10.537330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:10.564400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b691f02-a63a-472d-97c1-8fd6bd1f2c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:10.565026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b691f02-a63a-472d-97c1-8fd6bd1f2c9f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:10.565304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b691f02-a63a-472d-97c1-8fd6bd1f2c9f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:10.565571 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b691f02-a63a-472d-97c1-8fd6bd1f2c9f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:10.566143 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b691f02-a63a-472d-97c1-8fd6bd1f2c9f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:10.567506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b691f02-a63a-472d-97c1-8fd6bd1f2c9f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-8b691f02-a63a-472d-97c1-8fd6bd1f2c9f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:10.576926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:10.577918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:10.578810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:10.578810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:10.578810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:10.578810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:10.578810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:10.578810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:10.578810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:10.578810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:10.578810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:10.578810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:10.578810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:10.578810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:10.578810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 19 14:40:10.578810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:10.582885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 456/921] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:40:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:10.584293 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c1f412-8c95-404d-91d5-c177f0bd1009 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:10.644307 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:10.692954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c1f412-8c95-404d-91d5-c177f0bd1009 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:10.730911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ee280-eeb1-4955-8c51-885bc7b24051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:10.731549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ee280-eeb1-4955-8c51-885bc7b24051 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:10.731844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ee280-eeb1-4955-8c51-885bc7b24051 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:10.732125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ee280-eeb1-4955-8c51-885bc7b24051 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:10.732755 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ee280-eeb1-4955-8c51-885bc7b24051 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:10.733057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ee280-eeb1-4955-8c51-885bc7b24051 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:10.734406 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ee280-eeb1-4955-8c51-885bc7b24051 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:10.743917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 466/922] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:40:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:10.745292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b691f02-a63a-472d-97c1-8fd6bd1f2c9f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:10.798895 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca7e92-79e1-4725-ae7b-66e8e3ad9fac None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:10.799211 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca7e92-79e1-4725-ae7b-66e8e3ad9fac None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:10.799441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca7e92-79e1-4725-ae7b-66e8e3ad9fac None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:10.801339 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 467/923] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:40:10.807486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d32336-9d50-4995-9ee0-23844713814a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:10.807759 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d32336-9d50-4995-9ee0-23844713814a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:10.808061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d32336-9d50-4995-9ee0-23844713814a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:10.829830 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d1d32336-9d50-4995-9ee0-23844713814a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:10.831141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d1d32336-9d50-4995-9ee0-23844713814a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:10.866426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b691f02-a63a-472d-97c1-8fd6bd1f2c9f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:10.903571 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bd892-8758-4680-9b14-ccd9da6f43bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:10.904315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bd892-8758-4680-9b14-ccd9da6f43bd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:10.904506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bd892-8758-4680-9b14-ccd9da6f43bd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:10.904759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bd892-8758-4680-9b14-ccd9da6f43bd None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:10.905367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bd892-8758-4680-9b14-ccd9da6f43bd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:10.905664 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bd892-8758-4680-9b14-ccd9da6f43bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:10.907025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bd892-8758-4680-9b14-ccd9da6f43bd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:10.912689 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 457/924] 10.222.0.60 () {56 vars in 1114 bytes} [Mon Sep 19 14:40:10 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:10.920009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 468/925] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:40:10.921091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bd892-8758-4680-9b14-ccd9da6f43bd None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:10.960549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:11.035451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bd892-8758-4680-9b14-ccd9da6f43bd None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:11.073275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d9e6f-f823-4d8a-93f6-026b4b909c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:11.073275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d9e6f-f823-4d8a-93f6-026b4b909c1b None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:11.073275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d9e6f-f823-4d8a-93f6-026b4b909c1b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:11.073275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d9e6f-f823-4d8a-93f6-026b4b909c1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:11.073709 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d9e6f-f823-4d8a-93f6-026b4b909c1b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:11.075108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d9e6f-f823-4d8a-93f6-026b4b909c1b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:11.103244 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 458/926] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:40:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:40:11.220372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d32336-9d50-4995-9ee0-23844713814a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:11.277286 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb19386-6d88-43a8-bad5-b9ed5fc16c85 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:11.277667 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb19386-6d88-43a8-bad5-b9ed5fc16c85 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:11.277868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb19386-6d88-43a8-bad5-b9ed5fc16c85 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:11.279311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 459/927] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:11.279944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:11.286026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf90e5-5afb-42bf-ab2f-782748c7c6b7 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:11.286320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf90e5-5afb-42bf-ab2f-782748c7c6b7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:11.286719 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf90e5-5afb-42bf-ab2f-782748c7c6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:11.312903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d8cf90e5-5afb-42bf-ab2f-782748c7c6b7 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:11.314396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d8cf90e5-5afb-42bf-ab2f-782748c7c6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:11.346534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d32336-9d50-4995-9ee0-23844713814a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:11.397679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6897d-b8ee-45db-88a2-1c4d38f4f4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:11.398313 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6897d-b8ee-45db-88a2-1c4d38f4f4d0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:11.398593 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6897d-b8ee-45db-88a2-1c4d38f4f4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:11.398869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc6897d-b8ee-45db-88a2-1c4d38f4f4d0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:11.400318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 460/928] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:40:11.414044 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc6897d-b8ee-45db-88a2-1c4d38f4f4d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-2fc6897d-b8ee-45db-88a2-1c4d38f4f4d0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:40:11.418946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:11.420019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:11.420882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:11.420882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:11.420882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:11.420882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:11.420882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:40:11.420882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:40:11.420882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:40:11.420882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:40:11.420882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:40:11.420882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:40:11.420882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 14:40:11.420882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:11.420882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 469/929] 10.222.0.60 () {56 vars in 1120 bytes} [Mon Sep 19 14:40:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:11.426292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf90e5-5afb-42bf-ab2f-782748c7c6b7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:11.472309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:11.535972 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf90e5-5afb-42bf-ab2f-782748c7c6b7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:11.576892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6831b4-809c-4782-bcfe-4d45a9884f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:11.577509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6831b4-809c-4782-bcfe-4d45a9884f4f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:11.577790 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6831b4-809c-4782-bcfe-4d45a9884f4f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:11.578064 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6831b4-809c-4782-bcfe-4d45a9884f4f None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:11.578722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6831b4-809c-4782-bcfe-4d45a9884f4f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:11.579023 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6831b4-809c-4782-bcfe-4d45a9884f4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:11.580447 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6831b4-809c-4782-bcfe-4d45a9884f4f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:11.589142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 461/930] 10.222.0.60 () {56 vars in 1115 bytes} [Mon Sep 19 14:40:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:11.596919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6897d-b8ee-45db-88a2-1c4d38f4f4d0 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:11.670088 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6831b4-809c-4782-bcfe-4d45a9884f4f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:11.708234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc6897d-b8ee-45db-88a2-1c4d38f4f4d0 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:11.716745 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:11.745519 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2dd88-cb38-48b1-b974-a92bd97ac9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:11.746074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2dd88-cb38-48b1-b974-a92bd97ac9bc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:11.746355 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2dd88-cb38-48b1-b974-a92bd97ac9bc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:11.746630 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2dd88-cb38-48b1-b974-a92bd97ac9bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:11.757545 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2dd88-cb38-48b1-b974-a92bd97ac9bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:11.759008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2dd88-cb38-48b1-b974-a92bd97ac9bc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:11.760192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:11.777290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6831b4-809c-4782-bcfe-4d45a9884f4f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-aeb2dd88-cb38-48b1-b974-a92bd97ac9bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:11.777869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:11.778906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:11.779801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:11.779801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:11.779801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:11.786394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 470/931] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:11.787587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2dd88-cb38-48b1-b974-a92bd97ac9bc None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:11.812587 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683be6c9-79f8-4927-8c41-e6646dd01d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:11.813222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683be6c9-79f8-4927-8c41-e6646dd01d4f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:11.813531 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683be6c9-79f8-4927-8c41-e6646dd01d4f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:11.813770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683be6c9-79f8-4927-8c41-e6646dd01d4f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:11.814351 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683be6c9-79f8-4927-8c41-e6646dd01d4f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:11.815726 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683be6c9-79f8-4927-8c41-e6646dd01d4f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-683be6c9-79f8-4927-8c41-e6646dd01d4f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:11.824969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:11.826597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:11.827473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:11.827473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:11.827473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:11.827473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:11.827473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:11.827473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:11.827473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:11.827473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:11.827473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:11.827473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:11.827473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:11.827473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:11.827473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 14:40:11.827473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:11.827473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 462/932] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Sep 19 14:40:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:11.832672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683be6c9-79f8-4927-8c41-e6646dd01d4f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:11.914016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2dd88-cb38-48b1-b974-a92bd97ac9bc None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:11.958264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4710d7c-66ff-4a1c-9cb3-f1e71855c606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:11.958910 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4710d7c-66ff-4a1c-9cb3-f1e71855c606 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:11.959191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4710d7c-66ff-4a1c-9cb3-f1e71855c606 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:11.959463 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4710d7c-66ff-4a1c-9cb3-f1e71855c606 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:11.960424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4710d7c-66ff-4a1c-9cb3-f1e71855c606 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:11.960736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4710d7c-66ff-4a1c-9cb3-f1e71855c606 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:11.962291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4710d7c-66ff-4a1c-9cb3-f1e71855c606 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:11.973101 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683be6c9-79f8-4927-8c41-e6646dd01d4f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:11.981431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 471/933] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:11.987868 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4710d7c-66ff-4a1c-9cb3-f1e71855c606 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:12.010789 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:12.032019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6134b-7ec5-49d4-9a63-353dc31b1c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:12.032867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6134b-7ec5-49d4-9a63-353dc31b1c8a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:12.033313 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6134b-7ec5-49d4-9a63-353dc31b1c8a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:12.033725 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6134b-7ec5-49d4-9a63-353dc31b1c8a None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:12.034489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6134b-7ec5-49d4-9a63-353dc31b1c8a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:12.034928 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6134b-7ec5-49d4-9a63-353dc31b1c8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:12.036987 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6134b-7ec5-49d4-9a63-353dc31b1c8a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:12.044720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 463/934] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Sep 19 14:40:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:12.060732 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6134b-7ec5-49d4-9a63-353dc31b1c8a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:12.079551 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:12.120550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4710d7c-66ff-4a1c-9cb3-f1e71855c606 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:12.166627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c966d2d-06c8-45c3-841d-872bb48c1622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:12.167853 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c966d2d-06c8-45c3-841d-872bb48c1622 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:12.168332 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c966d2d-06c8-45c3-841d-872bb48c1622 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:12.169649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c966d2d-06c8-45c3-841d-872bb48c1622 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:12.176252 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6134b-7ec5-49d4-9a63-353dc31b1c8a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:12.180350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c966d2d-06c8-45c3-841d-872bb48c1622 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:12.182005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c966d2d-06c8-45c3-841d-872bb48c1622 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-2c966d2d-06c8-45c3-841d-872bb48c1622 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:40:12.198358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:12.199366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:12.200392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:12.200392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:12.200392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:40:12.200392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:40:12.200392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:40:12.200392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:40:12.200392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:40:12.200392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:40:12.200392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 14:40:12.200392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:12.206102 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 472/935] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Sep 19 14:40:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:12.206764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c966d2d-06c8-45c3-841d-872bb48c1622 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:12.227326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79194c7-ef20-4045-a38c-630578747298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:12.228320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79194c7-ef20-4045-a38c-630578747298 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:12.228805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79194c7-ef20-4045-a38c-630578747298 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:12.229281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79194c7-ef20-4045-a38c-630578747298 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:12.230404 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79194c7-ef20-4045-a38c-630578747298 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:12.230930 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79194c7-ef20-4045-a38c-630578747298 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:12.234103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79194c7-ef20-4045-a38c-630578747298 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:12.242074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 464/936] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Sep 19 14:40:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:12.248198 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79194c7-ef20-4045-a38c-630578747298 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:12.332072 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:12.337973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:12.364390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c966d2d-06c8-45c3-841d-872bb48c1622 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:12.365299 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79194c7-ef20-4045-a38c-630578747298 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:12.401653 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a3cd0b-a110-460f-acae-b863bdfeafef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:12.402269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3cd0b-a110-460f-acae-b863bdfeafef None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:12.402545 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3cd0b-a110-460f-acae-b863bdfeafef None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:12.402820 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a3cd0b-a110-460f-acae-b863bdfeafef None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:12.404085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3cd0b-a110-460f-acae-b863bdfeafef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:12.404443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3cd0b-a110-460f-acae-b863bdfeafef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:12.405068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74611dc2-e586-42de-8717-81b83bb067af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:12.405673 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74611dc2-e586-42de-8717-81b83bb067af None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:12.405950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74611dc2-e586-42de-8717-81b83bb067af None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:12.406046 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a3cd0b-a110-460f-acae-b863bdfeafef None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:12.406222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74611dc2-e586-42de-8717-81b83bb067af None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:12.406904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74611dc2-e586-42de-8717-81b83bb067af None admin] RBAC: Adding query filter params (interface=public, service_id=5ca1bed51fe44caf8c1dc9270f51657b, region_id=RegionOne) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:12.407208 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74611dc2-e586-42de-8717-81b83bb067af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:12.409540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74611dc2-e586-42de-8717-81b83bb067af None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:12.420776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 465/937] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Sep 19 14:40:12 2022] GET /identity/v3/endpoints?service_id=5ca1bed51fe44caf8c1dc9270f51657b&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:12.421328 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c34643-6e7f-465f-8011-0f4e5524d64d None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:12.421617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c34643-6e7f-465f-8011-0f4e5524d64d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:12.422459 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c34643-6e7f-465f-8011-0f4e5524d64d None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:12.423887 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 466/938] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:12.424536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c34643-6e7f-465f-8011-0f4e5524d64d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:12.430609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 473/939] 10.222.0.60 () {56 vars in 1149 bytes} [Mon Sep 19 14:40:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:12.431877 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c72fc-c148-4f76-a4df-9d745d79d984 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:12.432163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c72fc-c148-4f76-a4df-9d745d79d984 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:12.432435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c72fc-c148-4f76-a4df-9d745d79d984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:12.451605 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-669c72fc-c148-4f76-a4df-9d745d79d984 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:12.452862 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-669c72fc-c148-4f76-a4df-9d745d79d984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:12.529784 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 474/940] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:12.530915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c72fc-c148-4f76-a4df-9d745d79d984 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:12.551557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c34643-6e7f-465f-8011-0f4e5524d64d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:12.601999 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:12.604741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c9fb7-6c2f-4454-823d-0e9d9b20184c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:12.605599 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c9fb7-6c2f-4454-823d-0e9d9b20184c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:12.606516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c9fb7-6c2f-4454-823d-0e9d9b20184c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:12.607013 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c9fb7-6c2f-4454-823d-0e9d9b20184c None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:12.608674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c9fb7-6c2f-4454-823d-0e9d9b20184c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:12.611187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c9fb7-6c2f-4454-823d-0e9d9b20184c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:12.619781 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 467/941] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Sep 19 14:40:12 2022] GET /identity/v3/services => generated 1497 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:40:12.652264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669c72fc-c148-4f76-a4df-9d745d79d984 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:12.712858 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388c985-63c2-47fe-9580-4819ab24e814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:12.713514 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388c985-63c2-47fe-9580-4819ab24e814 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:12.713789 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388c985-63c2-47fe-9580-4819ab24e814 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:12.714103 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388c985-63c2-47fe-9580-4819ab24e814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:12.730694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388c985-63c2-47fe-9580-4819ab24e814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:12.732939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388c985-63c2-47fe-9580-4819ab24e814 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-9388c985-63c2-47fe-9580-4819ab24e814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:12.754766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:12.755811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:12.756756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:12.756756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:12.756756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:12.756756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 475/942] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:12.763909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c9fb7-6c2f-4454-823d-0e9d9b20184c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:12.769012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:12.832768 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388c985-63c2-47fe-9580-4819ab24e814 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:12.890223 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c9fb7-6c2f-4454-823d-0e9d9b20184c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:12.931363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718a7e2-6332-4ef5-b276-e291e1acdd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:12.932845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718a7e2-6332-4ef5-b276-e291e1acdd7f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:12.933544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718a7e2-6332-4ef5-b276-e291e1acdd7f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:12.933994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718a7e2-6332-4ef5-b276-e291e1acdd7f None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:12.935591 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718a7e2-6332-4ef5-b276-e291e1acdd7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:12.936489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718a7e2-6332-4ef5-b276-e291e1acdd7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:12.938073 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718a7e2-6332-4ef5-b276-e291e1acdd7f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:12.955436 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 468/943] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:12.955643 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388c985-63c2-47fe-9580-4819ab24e814 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:12.962346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718a7e2-6332-4ef5-b276-e291e1acdd7f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:12.979857 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:12.998454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebf834-715b-4742-a682-98f736e08386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:12.999094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebf834-715b-4742-a682-98f736e08386 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:12.999369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebf834-715b-4742-a682-98f736e08386 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:12.999634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebf834-715b-4742-a682-98f736e08386 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:13.000252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebf834-715b-4742-a682-98f736e08386 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:13.001716 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebf834-715b-4742-a682-98f736e08386 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-50ebf834-715b-4742-a682-98f736e08386 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:13.011414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:13.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:13.013504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:13.013504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:13.013504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:13.013504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:13.013504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:13.013504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:13.013504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:13.013504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:13.013504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:13.013504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:13.013504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:13.013504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:13.013504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 14:40:13.013504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:13.018057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 476/944] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:40:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:13.019364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebf834-715b-4742-a682-98f736e08386 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:13.047223 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:13.096593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718a7e2-6332-4ef5-b276-e291e1acdd7f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:13.136658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80acfb-3830-4a85-a966-cf0367c56b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:13.138293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80acfb-3830-4a85-a966-cf0367c56b4a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:13.138725 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80acfb-3830-4a85-a966-cf0367c56b4a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:13.139109 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80acfb-3830-4a85-a966-cf0367c56b4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:13.146575 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80acfb-3830-4a85-a966-cf0367c56b4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:13.148248 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80acfb-3830-4a85-a966-cf0367c56b4a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:13.160421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-0a80acfb-3830-4a85-a966-cf0367c56b4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:13.160421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:13.160421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:13.160421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:13.160421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:13.160421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:13.160421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:13.160421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:13.160421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:13.160421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:13.160421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:13.160421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:13.160421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:40:13.160421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:40:13.160421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:13.161613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:13.162751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:40:13.162751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:40:13.162751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:40:13.162751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:40:13.162751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:40:13.162751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:40:13.162751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:13.162751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:13.168722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 469/945] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:40:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:13.170865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80acfb-3830-4a85-a966-cf0367c56b4a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:13.176322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebf834-715b-4742-a682-98f736e08386 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:13.210314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ead8c-08fd-4d9e-85b5-c65ec6e5d04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:13.210927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ead8c-08fd-4d9e-85b5-c65ec6e5d04a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:13.211208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ead8c-08fd-4d9e-85b5-c65ec6e5d04a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:13.211480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ead8c-08fd-4d9e-85b5-c65ec6e5d04a None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:13.212161 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ead8c-08fd-4d9e-85b5-c65ec6e5d04a None admin] RBAC: Adding query filter params (name=compute) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:13.212463 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ead8c-08fd-4d9e-85b5-c65ec6e5d04a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:13.213833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ead8c-08fd-4d9e-85b5-c65ec6e5d04a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:13.221060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 477/946] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Sep 19 14:40:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:13.228889 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ead8c-08fd-4d9e-85b5-c65ec6e5d04a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:13.285713 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:13.289367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80acfb-3830-4a85-a966-cf0367c56b4a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:13.317301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:13.332854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588b66f-0e47-413a-bbbb-1c8f7e4c8ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:13.333906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588b66f-0e47-413a-bbbb-1c8f7e4c8ece None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:13.334426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588b66f-0e47-413a-bbbb-1c8f7e4c8ece None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:13.334958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588b66f-0e47-413a-bbbb-1c8f7e4c8ece None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:13.336014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588b66f-0e47-413a-bbbb-1c8f7e4c8ece None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:13.336556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588b66f-0e47-413a-bbbb-1c8f7e4c8ece None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:13.338259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6588b66f-0e47-413a-bbbb-1c8f7e4c8ece None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:13.355945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ead8c-08fd-4d9e-85b5-c65ec6e5d04a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:13.357188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 470/947] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:40:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:13.359363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588b66f-0e47-413a-bbbb-1c8f7e4c8ece None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:13.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0fc71-bf90-49e5-b548-b5da4c1591fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:13.402666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0fc71-bf90-49e5-b548-b5da4c1591fe None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:13.402947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0fc71-bf90-49e5-b548-b5da4c1591fe None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:13.403227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0fc71-bf90-49e5-b548-b5da4c1591fe None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:13.404444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0fc71-bf90-49e5-b548-b5da4c1591fe None admin] RBAC: Adding query filter params (type=compute) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:13.404746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0fc71-bf90-49e5-b548-b5da4c1591fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:13.407392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0fc71-bf90-49e5-b548-b5da4c1591fe None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:13.419855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 478/948] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Sep 19 14:40:13 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:13.439885 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f355ea-39eb-405b-a849-2d25b18db7f6 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:13.440230 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f355ea-39eb-405b-a849-2d25b18db7f6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:13.440451 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f355ea-39eb-405b-a849-2d25b18db7f6 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:13.441992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 479/949] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:40:13.448326 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979c3c3-27cf-47ed-892c-665b7530fbb8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:13.448738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979c3c3-27cf-47ed-892c-665b7530fbb8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:13.449019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979c3c3-27cf-47ed-892c-665b7530fbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:13.471172 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b979c3c3-27cf-47ed-892c-665b7530fbb8 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:13.472634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b979c3c3-27cf-47ed-892c-665b7530fbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:13.495414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588b66f-0e47-413a-bbbb-1c8f7e4c8ece None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:13.534130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe6df2-2cdb-42c1-9c91-10fe0947bc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:13.534803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe6df2-2cdb-42c1-9c91-10fe0947bc59 None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:13.535077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe6df2-2cdb-42c1-9c91-10fe0947bc59 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:13.535348 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe6df2-2cdb-42c1-9c91-10fe0947bc59 None admin] PATH_INFO: `/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/0fefb7fa6f484341a19a1a8bee31b116/roles/b88a5181c4564a8ca1acebcbd45465c0` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:13.537656 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe6df2-2cdb-42c1-9c91-10fe0947bc59 None admin] RBAC: Authorizing `identity:create_grant(project_id=7dee98edf19d478ca4f746ff85579fba, user_id=0fefb7fa6f484341a19a1a8bee31b116, role_id=b88a5181c4564a8ca1acebcbd45465c0)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:13.539425 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe6df2-2cdb-42c1-9c91-10fe0947bc59 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:13.565530 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 471/950] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:40:13 2022] PUT /identity/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/0fefb7fa6f484341a19a1a8bee31b116/roles/b88a5181c4564a8ca1acebcbd45465c0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:40:13.571698 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 480/951] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:13.851268 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe6df2-2cdb-42c1-9c91-10fe0947bc59 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:13.902857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:13.961372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe6df2-2cdb-42c1-9c91-10fe0947bc59 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:13.998433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94136f-1004-4833-ace8-485f6cc58939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:13.999050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94136f-1004-4833-ace8-485f6cc58939 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:13.999331 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94136f-1004-4833-ace8-485f6cc58939 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:13.999602 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa94136f-1004-4833-ace8-485f6cc58939 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:14.000326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94136f-1004-4833-ace8-485f6cc58939 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:14.001769 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa94136f-1004-4833-ace8-485f6cc58939 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:14.004530 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: WARNING keystone.server.flask.common [None req-aa94136f-1004-4833-ace8-485f6cc58939 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 19 14:40:14.025377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.sql.core [None req-aa94136f-1004-4833-ace8-485f6cc58939 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 14:40:14.025377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 19 14:40:14.025377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [parameters: {'id': 'bfe966db825f4da8a43c91b59c1f07ee', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 19 14:40:14.025377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87872) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 14:40:14.043585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-aa94136f-1004-4833-ace8-485f6cc58939 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bfe966db825f4da8a43c91b59c1f07ee.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bfe966db825f4da8a43c91b59c1f07ee. Sep 19 14:40:14.043585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:14.043585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 14:40:14.043585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 14:40:14.043585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 14:40:14.043585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 14:40:14.043585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 14:40:14.043585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 14:40:14.043585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 14:40:14.043585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application conn.query(q) Sep 19 14:40:14.043585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 14:40:14.043585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 14:40:14.043585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application result.read() Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 19 14:40:14.045658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application next(self.gen) Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application yield resource Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application next(self.gen) Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise self.value Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application session.commit() Sep 19 14:40:14.047908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self.session.flush() Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application compat.raise_( Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 14:40:14.052390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 14:40:14.054557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 14:40:14.054557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 14:40:14.054557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 14:40:14.054557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 14:40:14.054557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 19 14:40:14.054557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 19 14:40:14.054557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 14:40:14.054557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 14:40:14.054557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 14:40:14.054557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 14:40:14.054557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 14:40:14.054557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 14:40:14.054557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 14:40:14.054557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 14:40:14.054557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application conn.query(q) Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application result.read() Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 14:40:14.056460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 14:40:14.058540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 14:40:14.058540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 14:40:14.058540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 14:40:14.058540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 14:40:14.058540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 14:40:14.058540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 14:40:14.058540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 19 14:40:14.058540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 19 14:40:14.058540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [parameters: {'id': 'bfe966db825f4da8a43c91b59c1f07ee', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 19 14:40:14.058540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 14:40:14.058540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:14.058540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:14.058540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:14.058540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:14.058540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:14.068414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:14.070561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:14.070561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 19 14:40:14.070561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 19 14:40:14.070561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:14.070561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:14.070561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 19 14:40:14.070561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 19 14:40:14.070561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bfe966db825f4da8a43c91b59c1f07ee. Sep 19 14:40:14.070561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:14.070561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 472/952] 10.222.0.60 () {60 vars in 1140 bytes} [Mon Sep 19 14:40:13 2022] POST /identity/v3/projects => generated 262 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:14.071915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979c3c3-27cf-47ed-892c-665b7530fbb8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:14.099997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:14.163252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979c3c3-27cf-47ed-892c-665b7530fbb8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:14.199692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00695936-9403-4dc2-8deb-dbb4b3a31ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:14.200873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00695936-9403-4dc2-8deb-dbb4b3a31ec0 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:14.201177 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00695936-9403-4dc2-8deb-dbb4b3a31ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:14.201406 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00695936-9403-4dc2-8deb-dbb4b3a31ec0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:14.208525 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00695936-9403-4dc2-8deb-dbb4b3a31ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:14.210016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00695936-9403-4dc2-8deb-dbb4b3a31ec0 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:14.221682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-00695936-9403-4dc2-8deb-dbb4b3a31ec0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:14.221682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:14.221682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:14.221682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:14.221682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:14.221682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:14.221682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:14.221682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:14.221682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:14.221682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:14.221682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:14.221682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:14.221682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:40:14.221682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:40:14.221682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:14.223947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:14.226058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:40:14.226058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:40:14.226058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:40:14.226058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:40:14.226058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:40:14.226058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:40:14.226058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:14.226058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:14.226058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 481/953] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:40:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:14.228929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94136f-1004-4833-ace8-485f6cc58939 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:14.335929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:14.338986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa94136f-1004-4833-ace8-485f6cc58939 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:14.375689 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf8c7a-d866-4a8c-9d4b-0f6443f07a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:14.376332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf8c7a-d866-4a8c-9d4b-0f6443f07a30 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:14.376607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf8c7a-d866-4a8c-9d4b-0f6443f07a30 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:14.376888 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf8c7a-d866-4a8c-9d4b-0f6443f07a30 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:14.377547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf8c7a-d866-4a8c-9d4b-0f6443f07a30 None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:14.377843 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf8c7a-d866-4a8c-9d4b-0f6443f07a30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:14.379491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf8c7a-d866-4a8c-9d4b-0f6443f07a30 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:14.402720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 473/954] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Sep 19 14:40:14 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:15.001361 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97d7ab-d258-44d0-95a5-10ea39beafe2 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:15.001745 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97d7ab-d258-44d0-95a5-10ea39beafe2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:15.001921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97d7ab-d258-44d0-95a5-10ea39beafe2 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:15.003493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 482/955] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:15.008897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cd9f7-912f-48d3-b3ed-6add3c8f7264 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:15.009503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cd9f7-912f-48d3-b3ed-6add3c8f7264 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:15.009503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cd9f7-912f-48d3-b3ed-6add3c8f7264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:15.029391 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b89cd9f7-912f-48d3-b3ed-6add3c8f7264 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:15.031028 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b89cd9f7-912f-48d3-b3ed-6add3c8f7264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:15.111461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 474/956] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:15.388180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97d7ab-d258-44d0-95a5-10ea39beafe2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:15.496922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:15.500823 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97d7ab-d258-44d0-95a5-10ea39beafe2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:15.537039 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53140485-dfbd-4a3e-808b-a9ecd17023ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:15.537667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53140485-dfbd-4a3e-808b-a9ecd17023ba None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:15.538050 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53140485-dfbd-4a3e-808b-a9ecd17023ba None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:15.538330 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53140485-dfbd-4a3e-808b-a9ecd17023ba None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:15.538909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53140485-dfbd-4a3e-808b-a9ecd17023ba None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:15.540291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53140485-dfbd-4a3e-808b-a9ecd17023ba None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-53140485-dfbd-4a3e-808b-a9ecd17023ba None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:15.549047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:15.551462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:15.552461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:15.552461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:15.552461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:15.552461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:15.552461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:15.552461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:15.552461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:15.552461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:15.552461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:15.552461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:15.552461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:15.552461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:15.552461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 14:40:15.552461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:15.552461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 483/957] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Sep 19 14:40:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:15.556394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cd9f7-912f-48d3-b3ed-6add3c8f7264 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:15.656641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:15.666649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cd9f7-912f-48d3-b3ed-6add3c8f7264 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:15.707408 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c632b-07bd-4d6d-a0e0-e7a91627ff09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:15.708061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c632b-07bd-4d6d-a0e0-e7a91627ff09 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:15.708343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c632b-07bd-4d6d-a0e0-e7a91627ff09 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:15.708618 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c632b-07bd-4d6d-a0e0-e7a91627ff09 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:15.709252 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c632b-07bd-4d6d-a0e0-e7a91627ff09 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:15.709557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c632b-07bd-4d6d-a0e0-e7a91627ff09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:15.711019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c632b-07bd-4d6d-a0e0-e7a91627ff09 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:15.717581 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 475/958] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Sep 19 14:40:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:15.725002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53140485-dfbd-4a3e-808b-a9ecd17023ba None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:15.806205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29378cb-5b2d-4bb3-8e72-bdff2589d76c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:15.806629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29378cb-5b2d-4bb3-8e72-bdff2589d76c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:15.806775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29378cb-5b2d-4bb3-8e72-bdff2589d76c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:15.808396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 476/959] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:40:15.814387 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4dc4a-a8f2-4867-94ce-a11f67b86887 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:15.814766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4dc4a-a8f2-4867-94ce-a11f67b86887 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:15.819652 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b4dc4a-a8f2-4867-94ce-a11f67b86887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:15.834779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a1b4dc4a-a8f2-4867-94ce-a11f67b86887 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:15.836282 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a1b4dc4a-a8f2-4867-94ce-a11f67b86887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:15.872652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53140485-dfbd-4a3e-808b-a9ecd17023ba None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:15.922824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 477/960] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:40:15.922824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f410f7c-4ac4-421b-a5cc-6defff2b422e None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:15.922824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f410f7c-4ac4-421b-a5cc-6defff2b422e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:15.922824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f410f7c-4ac4-421b-a5cc-6defff2b422e None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:15.923595 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 478/961] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:15.929262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e14c3-614f-4dca-879d-05a50a48aa15 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:15.929544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e14c3-614f-4dca-879d-05a50a48aa15 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:15.929814 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e14c3-614f-4dca-879d-05a50a48aa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:15.932548 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7175d6-dadd-4818-b632-24be0c30143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:15.933594 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7175d6-dadd-4818-b632-24be0c30143a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:15.934109 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7175d6-dadd-4818-b632-24be0c30143a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:15.935235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7175d6-dadd-4818-b632-24be0c30143a None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:15.936193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7175d6-dadd-4818-b632-24be0c30143a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:15.936723 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7175d6-dadd-4818-b632-24be0c30143a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:15.939675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7175d6-dadd-4818-b632-24be0c30143a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:15.946684 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 484/962] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Sep 19 14:40:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:15.952013 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7175d6-dadd-4818-b632-24be0c30143a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:15.962580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8a4e14c3-614f-4dca-879d-05a50a48aa15 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:15.962734 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8a4e14c3-614f-4dca-879d-05a50a48aa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:15.987808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:16.039046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 479/963] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:40:16.079795 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7175d6-dadd-4818-b632-24be0c30143a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:16.117705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65252d73-b27e-45f0-9f24-58ea6033763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:16.118315 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65252d73-b27e-45f0-9f24-58ea6033763b None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:16.118590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65252d73-b27e-45f0-9f24-58ea6033763b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:16.118863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65252d73-b27e-45f0-9f24-58ea6033763b None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:16.119516 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65252d73-b27e-45f0-9f24-58ea6033763b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:16.120891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65252d73-b27e-45f0-9f24-58ea6033763b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:16.142606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 485/964] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:40:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:40:16.216858 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e14c3-614f-4dca-879d-05a50a48aa15 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:16.234533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:16.340446 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65252d73-b27e-45f0-9f24-58ea6033763b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:16.366785 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e14c3-614f-4dca-879d-05a50a48aa15 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:16.426375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3440f1-7047-4a96-9c12-80b8109df6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:16.427530 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3440f1-7047-4a96-9c12-80b8109df6cd None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:16.428165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3440f1-7047-4a96-9c12-80b8109df6cd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:16.428638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3440f1-7047-4a96-9c12-80b8109df6cd None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:16.429732 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3440f1-7047-4a96-9c12-80b8109df6cd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:16.432272 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3440f1-7047-4a96-9c12-80b8109df6cd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:16.446536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:16.458675 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 480/965] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Sep 19 14:40:16 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:40:16.468898 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65252d73-b27e-45f0-9f24-58ea6033763b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:16.519145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b7d416-8b1f-430d-9646-77addcfa2f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:16.520938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7d416-8b1f-430d-9646-77addcfa2f87 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:16.521536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7d416-8b1f-430d-9646-77addcfa2f87 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:16.522067 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b7d416-8b1f-430d-9646-77addcfa2f87 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:16.537472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b7d416-8b1f-430d-9646-77addcfa2f87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-e7b7d416-8b1f-430d-9646-77addcfa2f87 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:40:16.543356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:16.544493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:16.545349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:16.545349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:16.545349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:16.545349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:16.545349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:40:16.545349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:40:16.545349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:40:16.545349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:40:16.545349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:40:16.545349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:40:16.545349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 14:40:16.545349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:16.548458 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 486/966] 10.222.0.60 () {56 vars in 1120 bytes} [Mon Sep 19 14:40:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:16.555085 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3440f1-7047-4a96-9c12-80b8109df6cd None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:16.642842 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:16.684268 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374fb4b-e00a-4c9a-96ce-e1cf3316d461 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:16.684595 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374fb4b-e00a-4c9a-96ce-e1cf3316d461 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:16.684855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f374fb4b-e00a-4c9a-96ce-e1cf3316d461 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:16.686487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 487/967] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:40:16.688095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3440f1-7047-4a96-9c12-80b8109df6cd None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:16.692773 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0bbc9-46ed-4b9a-ac98-85729a485736 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:16.693067 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0bbc9-46ed-4b9a-ac98-85729a485736 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:16.693324 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0bbc9-46ed-4b9a-ac98-85729a485736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:16.720304 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ebb0bbc9-46ed-4b9a-ac98-85729a485736 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:16.721897 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ebb0bbc9-46ed-4b9a-ac98-85729a485736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:16.735286 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b76fd-f2b5-4a72-b339-426d3627d826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:16.735601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b76fd-f2b5-4a72-b339-426d3627d826 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:16.735849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b76fd-f2b5-4a72-b339-426d3627d826 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:16.736117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b76fd-f2b5-4a72-b339-426d3627d826 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:16.736781 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b76fd-f2b5-4a72-b339-426d3627d826 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:16.737075 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b76fd-f2b5-4a72-b339-426d3627d826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:16.738449 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391b76fd-f2b5-4a72-b339-426d3627d826 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:16.747118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 481/968] 10.222.0.60 () {56 vars in 1115 bytes} [Mon Sep 19 14:40:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:16.752599 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b76fd-f2b5-4a72-b339-426d3627d826 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:16.815205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 488/969] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:16.865768 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b76fd-f2b5-4a72-b339-426d3627d826 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:16.896738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:16.902966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26669cb1-f3ac-4314-8653-caa0e1e4493f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:16.904191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26669cb1-f3ac-4314-8653-caa0e1e4493f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:16.904471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26669cb1-f3ac-4314-8653-caa0e1e4493f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:16.904741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26669cb1-f3ac-4314-8653-caa0e1e4493f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:16.916302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26669cb1-f3ac-4314-8653-caa0e1e4493f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:16.917757 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26669cb1-f3ac-4314-8653-caa0e1e4493f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-26669cb1-f3ac-4314-8653-caa0e1e4493f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:16.933514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:16.935180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:16.936153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:16.936153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:16.936153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:16.939898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 482/970] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:16.941254 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0bbc9-46ed-4b9a-ac98-85729a485736 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:17.020932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:17.058889 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0bbc9-46ed-4b9a-ac98-85729a485736 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:17.087082 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26669cb1-f3ac-4314-8653-caa0e1e4493f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:17.096349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f62be-6c16-4ebb-97f3-88e82cd1a6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:17.097177 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f62be-6c16-4ebb-97f3-88e82cd1a6ae None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:17.097622 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f62be-6c16-4ebb-97f3-88e82cd1a6ae None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:17.098044 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f62be-6c16-4ebb-97f3-88e82cd1a6ae None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:17.098825 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f62be-6c16-4ebb-97f3-88e82cd1a6ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:17.099266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f62be-6c16-4ebb-97f3-88e82cd1a6ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:17.100797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f62be-6c16-4ebb-97f3-88e82cd1a6ae None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:17.113944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 489/971] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:17.120769 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f62be-6c16-4ebb-97f3-88e82cd1a6ae None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:17.218137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26669cb1-f3ac-4314-8653-caa0e1e4493f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:17.255697 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020140b-92ba-4da5-b073-d1b3aa7edade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:17.256482 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020140b-92ba-4da5-b073-d1b3aa7edade None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:17.256861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020140b-92ba-4da5-b073-d1b3aa7edade None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:17.257233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3020140b-92ba-4da5-b073-d1b3aa7edade None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:17.257356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f62be-6c16-4ebb-97f3-88e82cd1a6ae None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:17.258111 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020140b-92ba-4da5-b073-d1b3aa7edade None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:17.259699 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3020140b-92ba-4da5-b073-d1b3aa7edade None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:17.276465 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.sql.core [None req-3020140b-92ba-4da5-b073-d1b3aa7edade None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 14:40:17.276465 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 19 14:40:17.276465 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [parameters: {'user_id': '81c17f1318c5422ead5bb14d9292f5da', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 19 14:40:17.276465 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87872) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 19 14:40:17.284060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-3020140b-92ba-4da5-b073-d1b3aa7edade None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 19 14:40:17.284060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:17.284060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 14:40:17.284060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 14:40:17.284060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 14:40:17.284060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 14:40:17.284060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 14:40:17.284060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 14:40:17.284060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 14:40:17.284060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application conn.query(q) Sep 19 14:40:17.284060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 14:40:17.284060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 14:40:17.284060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 14:40:17.284060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application result.read() Sep 19 14:40:17.284060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 14:40:17.284964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application next(self.gen) Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application yield resource Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application next(self.gen) Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self.session.rollback() Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self.force_reraise() Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise self.value Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application session.commit() Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 19 14:40:17.285946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._prepare_impl() Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self.session.flush() Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._flush(objects) Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application compat.raise_( Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application flush_context.execute() Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rec.execute(self) Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 19 14:40:17.286867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 19 14:40:17.287798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application result = self._query(query) Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application conn.query(q) Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application result.read() Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 19 14:40:17.290556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 19 14:40:17.298798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 19 14:40:17.298798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 19 14:40:17.298798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 19 14:40:17.298798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 19 14:40:17.298798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 19 14:40:17.298798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 19 14:40:17.298798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [parameters: {'user_id': '81c17f1318c5422ead5bb14d9292f5da', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 19 14:40:17.298798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 19 14:40:17.298798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:17.298798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:17.298798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:17.298798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:17.298798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:17.298798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:17.298798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 19 14:40:17.299815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 19 14:40:17.301241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 19 14:40:17.301241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 19 14:40:17.301241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 19 14:40:17.301241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 19 14:40:17.301241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 19 14:40:17.301241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:17.301241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 483/972] 10.222.0.60 () {60 vars in 1131 bytes} [Mon Sep 19 14:40:17 2022] POST /identity/v3/users => generated 160 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:17.301241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020140b-92ba-4da5-b073-d1b3aa7edade None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:17.305173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:17.311614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcaa0c2-22e0-4e8d-b398-e1a39b1cc02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:17.312683 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcaa0c2-22e0-4e8d-b398-e1a39b1cc02d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:17.313064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcaa0c2-22e0-4e8d-b398-e1a39b1cc02d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:17.313432 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcaa0c2-22e0-4e8d-b398-e1a39b1cc02d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:17.322171 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcaa0c2-22e0-4e8d-b398-e1a39b1cc02d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:17.323923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcaa0c2-22e0-4e8d-b398-e1a39b1cc02d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-0bcaa0c2-22e0-4e8d-b398-e1a39b1cc02d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:40:17.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:17.338425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:17.339392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:17.339392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:17.339392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:40:17.339392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:40:17.339392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:40:17.339392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:40:17.339392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:40:17.339392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:40:17.339392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 19 14:40:17.339392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:17.344904 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 490/973] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Sep 19 14:40:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:17.346331 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcaa0c2-22e0-4e8d-b398-e1a39b1cc02d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:17.355466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:17.426865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3020140b-92ba-4da5-b073-d1b3aa7edade None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:17.464159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43998a74-1de6-4422-aafd-35329245cc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:17.464932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43998a74-1de6-4422-aafd-35329245cc27 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:17.465318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43998a74-1de6-4422-aafd-35329245cc27 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:17.465662 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43998a74-1de6-4422-aafd-35329245cc27 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:17.473861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43998a74-1de6-4422-aafd-35329245cc27 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:17.475569 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43998a74-1de6-4422-aafd-35329245cc27 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:17.480819 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcaa0c2-22e0-4e8d-b398-e1a39b1cc02d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-43998a74-1de6-4422-aafd-35329245cc27 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:40:17.487295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:17.488308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:17.489300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:17.489300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:17.489300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:40:17.489300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:40:17.489300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:40:17.489300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:40:17.489300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:40:17.489300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:40:17.489300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 19 14:40:17.489300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:17.489300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 484/974] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:40:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:17.497720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43998a74-1de6-4422-aafd-35329245cc27 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:17.525392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782d693-a305-49ca-a268-3712be30fed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:17.526007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782d693-a305-49ca-a268-3712be30fed2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:17.526283 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782d693-a305-49ca-a268-3712be30fed2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:17.526612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782d693-a305-49ca-a268-3712be30fed2 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:17.527516 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782d693-a305-49ca-a268-3712be30fed2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:17.527699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782d693-a305-49ca-a268-3712be30fed2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:17.529234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782d693-a305-49ca-a268-3712be30fed2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:17.548518 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 491/975] 10.222.0.60 () {56 vars in 1149 bytes} [Mon Sep 19 14:40:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:17.554025 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782d693-a305-49ca-a268-3712be30fed2 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:17.615446 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:17.624497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43998a74-1de6-4422-aafd-35329245cc27 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:17.631814 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:17.662528 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38cece-38f7-4f04-a37a-93d8390f1d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:17.663382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38cece-38f7-4f04-a37a-93d8390f1d4c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:17.663849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38cece-38f7-4f04-a37a-93d8390f1d4c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:17.664290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38cece-38f7-4f04-a37a-93d8390f1d4c None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:17.665203 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38cece-38f7-4f04-a37a-93d8390f1d4c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:17.665675 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38cece-38f7-4f04-a37a-93d8390f1d4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:17.667471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38cece-38f7-4f04-a37a-93d8390f1d4c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:17.681785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782d693-a305-49ca-a268-3712be30fed2 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:17.707264 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 485/976] 10.222.0.60 () {56 vars in 1101 bytes} [Mon Sep 19 14:40:17 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:17.729843 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeff28-8574-497e-8847-dbed5f768cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:17.730732 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeff28-8574-497e-8847-dbed5f768cda None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:17.731164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeff28-8574-497e-8847-dbed5f768cda None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:17.732230 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aeff28-8574-497e-8847-dbed5f768cda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:17.809492 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aeff28-8574-497e-8847-dbed5f768cda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:17.811245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aeff28-8574-497e-8847-dbed5f768cda None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:17.867444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:17.933834 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 492/977] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:40:17 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:40:17.940306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38cece-38f7-4f04-a37a-93d8390f1d4c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:18.050542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38cece-38f7-4f04-a37a-93d8390f1d4c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:18.090452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ec26a-eb3a-4e18-be94-b60a5d49ffdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:18.091151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ec26a-eb3a-4e18-be94-b60a5d49ffdc None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:18.091435 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ec26a-eb3a-4e18-be94-b60a5d49ffdc None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:18.091710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50ec26a-eb3a-4e18-be94-b60a5d49ffdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:18.102433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ec26a-eb3a-4e18-be94-b60a5d49ffdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:18.103922 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50ec26a-eb3a-4e18-be94-b60a5d49ffdc None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c50ec26a-eb3a-4e18-be94-b60a5d49ffdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:18.120238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:18.121308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:18.127430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:18.127430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:18.127430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:18.127430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 486/978] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:18.129247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeff28-8574-497e-8847-dbed5f768cda None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:18.236316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aeff28-8574-497e-8847-dbed5f768cda None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:18.271786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014f5fa-9a8b-420e-92ac-da1fe34d0d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:18.272417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014f5fa-9a8b-420e-92ac-da1fe34d0d04 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:18.272700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014f5fa-9a8b-420e-92ac-da1fe34d0d04 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:18.272973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c014f5fa-9a8b-420e-92ac-da1fe34d0d04 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:18.273607 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014f5fa-9a8b-420e-92ac-da1fe34d0d04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:18.273910 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014f5fa-9a8b-420e-92ac-da1fe34d0d04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:18.275266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c014f5fa-9a8b-420e-92ac-da1fe34d0d04 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:18.287735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 493/979] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:18.293072 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ec26a-eb3a-4e18-be94-b60a5d49ffdc None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:18.312688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:18.402029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50ec26a-eb3a-4e18-be94-b60a5d49ffdc None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:18.441017 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b7b44-6104-4993-9a83-46e8785813bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:18.441649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b7b44-6104-4993-9a83-46e8785813bb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:18.441931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b7b44-6104-4993-9a83-46e8785813bb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:18.442206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b7b44-6104-4993-9a83-46e8785813bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:18.449110 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b7b44-6104-4993-9a83-46e8785813bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:18.450674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b7b44-6104-4993-9a83-46e8785813bb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:18.462118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-d62b7b44-6104-4993-9a83-46e8785813bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:18.462118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:18.462118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:18.462118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:18.462118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:18.462118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:18.462118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:18.462118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:18.462118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:18.462118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:18.462118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:18.462118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:18.462118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:40:18.462118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:40:18.462118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:18.463060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:18.464039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:40:18.464039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:40:18.464039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:40:18.464039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:40:18.464039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:40:18.464039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:40:18.464039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:18.464039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:18.467982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 487/980] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:40:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:18.469787 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014f5fa-9a8b-420e-92ac-da1fe34d0d04 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:18.481349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:18.519070 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d14499-608f-4676-86d8-4fdc99ecb2de None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:18.519463 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d14499-608f-4676-86d8-4fdc99ecb2de None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:18.519604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d14499-608f-4676-86d8-4fdc99ecb2de None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:18.521207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 488/981] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:18.525932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd44f2c-b389-4b32-8093-ba904e733438 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:18.526212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd44f2c-b389-4b32-8093-ba904e733438 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:18.526482 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd44f2c-b389-4b32-8093-ba904e733438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:18.547117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-cfd44f2c-b389-4b32-8093-ba904e733438 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:18.548773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cfd44f2c-b389-4b32-8093-ba904e733438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:18.577275 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c014f5fa-9a8b-420e-92ac-da1fe34d0d04 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:18.616169 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd197e8-1584-45c2-a01a-9862cf6695a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:18.616819 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd197e8-1584-45c2-a01a-9862cf6695a3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:18.617092 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd197e8-1584-45c2-a01a-9862cf6695a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:18.617519 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd197e8-1584-45c2-a01a-9862cf6695a3 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:18.618405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd197e8-1584-45c2-a01a-9862cf6695a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:18.618706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd197e8-1584-45c2-a01a-9862cf6695a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:18.620210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd197e8-1584-45c2-a01a-9862cf6695a3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:18.624966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 489/982] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:40:18.638658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 494/983] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:40:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:18.644687 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd44f2c-b389-4b32-8093-ba904e733438 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:18.654232 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:18.753330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd44f2c-b389-4b32-8093-ba904e733438 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:18.754955 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb493c-c774-42b2-a7e8-bbd8c468a638 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:18.755314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb493c-c774-42b2-a7e8-bbd8c468a638 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:18.755790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb493c-c774-42b2-a7e8-bbd8c468a638 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:18.758941 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 495/984] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:40:18.763847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7099534-e3d9-4497-b63d-ce7fa725b404 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:18.764194 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7099534-e3d9-4497-b63d-ce7fa725b404 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:18.764521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7099534-e3d9-4497-b63d-ce7fa725b404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:18.785860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e7099534-e3d9-4497-b63d-ce7fa725b404 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:18.787255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e7099534-e3d9-4497-b63d-ce7fa725b404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:18.797248 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3e906-91f7-4736-8372-b2c3e239c11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:18.797951 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3e906-91f7-4736-8372-b2c3e239c11f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:18.798296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3e906-91f7-4736-8372-b2c3e239c11f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:18.798637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3e906-91f7-4736-8372-b2c3e239c11f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:18.799434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3e906-91f7-4736-8372-b2c3e239c11f None admin] RBAC: Adding query filter params (role.id=b88a5181c4564a8ca1acebcbd45465c0, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=0fefb7fa6f484341a19a1a8bee31b116) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:18.799743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3e906-91f7-4736-8372-b2c3e239c11f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:18.801462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e3e906-91f7-4736-8372-b2c3e239c11f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:18.812461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 490/985] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:40:18 2022] GET /identity/v3/role_assignments?user.id=0fefb7fa6f484341a19a1a8bee31b116&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=b88a5181c4564a8ca1acebcbd45465c0 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:18.864538 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:18.871880 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 496/986] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:18 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:40:18.918800 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3e906-91f7-4736-8372-b2c3e239c11f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:18.996830 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:19.029574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3e906-91f7-4736-8372-b2c3e239c11f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:19.065222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399f32f-dc7c-4574-8cb7-3e3481eaa220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:19.066103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399f32f-dc7c-4574-8cb7-3e3481eaa220 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:19.066529 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399f32f-dc7c-4574-8cb7-3e3481eaa220 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:19.066945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399f32f-dc7c-4574-8cb7-3e3481eaa220 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:19.067689 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399f32f-dc7c-4574-8cb7-3e3481eaa220 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:19.069234 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399f32f-dc7c-4574-8cb7-3e3481eaa220 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-7399f32f-dc7c-4574-8cb7-3e3481eaa220 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:19.083374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:19.084480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:19.085372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:19.085372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:19.085372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:19.085372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:19.085372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:19.085372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:19.085372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:19.085372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:19.085372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:19.085372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:19.085372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:19.085372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:19.085372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 14:40:19.085372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:19.092399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 491/987] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Sep 19 14:40:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:19.094952 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7099534-e3d9-4497-b63d-ce7fa725b404 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:19.140385 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399f32f-dc7c-4574-8cb7-3e3481eaa220 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:19.202804 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7099534-e3d9-4497-b63d-ce7fa725b404 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:19.245671 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6308e-4ee6-4b75-9d60-b4e72815cf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:19.246287 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6308e-4ee6-4b75-9d60-b4e72815cf51 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:19.246570 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6308e-4ee6-4b75-9d60-b4e72815cf51 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:19.246847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6308e-4ee6-4b75-9d60-b4e72815cf51 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:19.248046 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6308e-4ee6-4b75-9d60-b4e72815cf51 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:19.248362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6308e-4ee6-4b75-9d60-b4e72815cf51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:19.249712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6308e-4ee6-4b75-9d60-b4e72815cf51 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:19.255035 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 497/988] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Sep 19 14:40:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:19.260658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6308e-4ee6-4b75-9d60-b4e72815cf51 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:19.267513 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399f32f-dc7c-4574-8cb7-3e3481eaa220 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:19.306489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f483a0-193a-4ae9-a0ce-fb288814723f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:19.307131 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f483a0-193a-4ae9-a0ce-fb288814723f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:19.307415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f483a0-193a-4ae9-a0ce-fb288814723f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:19.307689 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f483a0-193a-4ae9-a0ce-fb288814723f None admin] PATH_INFO: `/v3/services/compute` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:19.308377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f483a0-193a-4ae9-a0ce-fb288814723f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:19.310325 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f483a0-193a-4ae9-a0ce-fb288814723f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:19.312535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-31f483a0-193a-4ae9-a0ce-fb288814723f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:19.321161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:19.322412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:19.323898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:19.323898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:19.323898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:19.323898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:19.323898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:19.323898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:19.323898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:19.323898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:19.323898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:19.323898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:19.323898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:19.323898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:19.323898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 14:40:19.323898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:19.323898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 492/989] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:40:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:19.328833 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f483a0-193a-4ae9-a0ce-fb288814723f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:19.339419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:19.381189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6308e-4ee6-4b75-9d60-b4e72815cf51 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:19.420034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac214a41-f4cb-460a-b32d-a469b611338e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:19.420754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac214a41-f4cb-460a-b32d-a469b611338e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:19.420943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac214a41-f4cb-460a-b32d-a469b611338e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:19.421193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac214a41-f4cb-460a-b32d-a469b611338e None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:19.421812 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac214a41-f4cb-460a-b32d-a469b611338e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:19.422112 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac214a41-f4cb-460a-b32d-a469b611338e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:19.423473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac214a41-f4cb-460a-b32d-a469b611338e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:19.433482 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 498/990] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Sep 19 14:40:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:19.434836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac214a41-f4cb-460a-b32d-a469b611338e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:19.440770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f483a0-193a-4ae9-a0ce-fb288814723f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:19.476449 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d64fd-7c7f-4c25-926d-78346a3e03b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:19.476974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d64fd-7c7f-4c25-926d-78346a3e03b6 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:19.477543 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d64fd-7c7f-4c25-926d-78346a3e03b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:19.477543 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d64fd-7c7f-4c25-926d-78346a3e03b6 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:19.478140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d64fd-7c7f-4c25-926d-78346a3e03b6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:19.478436 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d64fd-7c7f-4c25-926d-78346a3e03b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:19.479797 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d64fd-7c7f-4c25-926d-78346a3e03b6 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:19.485581 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 493/991] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Sep 19 14:40:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:19.490860 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d64fd-7c7f-4c25-926d-78346a3e03b6 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:19.542388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac214a41-f4cb-460a-b32d-a469b611338e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:19.558114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:19.571297 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:19.583073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578da5b1-da13-4b95-b15c-ddcda4d26e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:19.583686 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578da5b1-da13-4b95-b15c-ddcda4d26e68 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:19.584003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578da5b1-da13-4b95-b15c-ddcda4d26e68 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:19.584278 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578da5b1-da13-4b95-b15c-ddcda4d26e68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:19.584992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578da5b1-da13-4b95-b15c-ddcda4d26e68 None admin] RBAC: Adding query filter params (interface=admin, service_id=5ca1bed51fe44caf8c1dc9270f51657b, region_id=RegionOne) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:19.585300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578da5b1-da13-4b95-b15c-ddcda4d26e68 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:19.586667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578da5b1-da13-4b95-b15c-ddcda4d26e68 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:19.592557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 499/992] 10.222.0.60 () {56 vars in 1249 bytes} [Mon Sep 19 14:40:19 2022] GET /identity/v3/endpoints?service_id=5ca1bed51fe44caf8c1dc9270f51657b&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:19.598198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578da5b1-da13-4b95-b15c-ddcda4d26e68 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:19.607197 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d64fd-7c7f-4c25-926d-78346a3e03b6 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:19.643948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceac366-be2e-4b71-ad9e-9c07f7aa6694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:19.644664 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceac366-be2e-4b71-ad9e-9c07f7aa6694 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:19.645020 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceac366-be2e-4b71-ad9e-9c07f7aa6694 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:19.645372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceac366-be2e-4b71-ad9e-9c07f7aa6694 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:19.646115 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceac366-be2e-4b71-ad9e-9c07f7aa6694 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:19.646492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceac366-be2e-4b71-ad9e-9c07f7aa6694 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:19.647976 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceac366-be2e-4b71-ad9e-9c07f7aa6694 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:19.658239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 494/993] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Sep 19 14:40:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:19.660027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceac366-be2e-4b71-ad9e-9c07f7aa6694 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:19.712370 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578da5b1-da13-4b95-b15c-ddcda4d26e68 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:19.752117 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c98d3-256f-4d12-a472-4b421c517036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:19.752759 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c98d3-256f-4d12-a472-4b421c517036 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:19.753038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c98d3-256f-4d12-a472-4b421c517036 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:19.753311 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c98d3-256f-4d12-a472-4b421c517036 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:19.753913 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c98d3-256f-4d12-a472-4b421c517036 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:19.755401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c98d3-256f-4d12-a472-4b421c517036 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:19.762652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 500/994] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Sep 19 14:40:19 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:40:19.768868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceac366-be2e-4b71-ad9e-9c07f7aa6694 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:19.808221 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:19.818438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17047e7b-3555-46aa-be36-cac843686568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:19.819081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17047e7b-3555-46aa-be36-cac843686568 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:19.819371 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17047e7b-3555-46aa-be36-cac843686568 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:19.819645 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17047e7b-3555-46aa-be36-cac843686568 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:19.820481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17047e7b-3555-46aa-be36-cac843686568 None admin] RBAC: Adding query filter params (interface=public, service_id=8c0c25e7b45c4634b8c185acd1e39562, region_id=RegionOne) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:19.820797 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17047e7b-3555-46aa-be36-cac843686568 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:19.822223 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17047e7b-3555-46aa-be36-cac843686568 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:19.834851 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 495/995] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Sep 19 14:40:19 2022] GET /identity/v3/endpoints?service_id=8c0c25e7b45c4634b8c185acd1e39562&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:19.835056 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c98d3-256f-4d12-a472-4b421c517036 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:19.899293 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:19.951110 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c98d3-256f-4d12-a472-4b421c517036 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:19.989750 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a148c-7db4-4b04-a57c-8f3fecf3d837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:19.990364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a148c-7db4-4b04-a57c-8f3fecf3d837 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:19.990644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a148c-7db4-4b04-a57c-8f3fecf3d837 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:19.990916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a148c-7db4-4b04-a57c-8f3fecf3d837 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:19.991512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a148c-7db4-4b04-a57c-8f3fecf3d837 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:19.992955 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a148c-7db4-4b04-a57c-8f3fecf3d837 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:20.003979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 501/996] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Sep 19 14:40:19 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:40:21.025346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e571c-09d9-4612-9f43-ce3b4b891419 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:21.025878 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e571c-09d9-4612-9f43-ce3b4b891419 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:21.025878 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e571c-09d9-4612-9f43-ce3b4b891419 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:21.027433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 496/997] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:21.032493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d564d3d-aa00-4a64-a0f4-4d04e9863e8a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:21.032904 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d564d3d-aa00-4a64-a0f4-4d04e9863e8a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:21.033065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d564d3d-aa00-4a64-a0f4-4d04e9863e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:21.051699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8d564d3d-aa00-4a64-a0f4-4d04e9863e8a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:21.052928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8d564d3d-aa00-4a64-a0f4-4d04e9863e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:21.129255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 502/998] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:21 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:21.395510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e571c-09d9-4612-9f43-ce3b4b891419 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:21.500919 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e571c-09d9-4612-9f43-ce3b4b891419 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:21.535464 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c437a1-81f6-4c50-8471-d80effd09c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:21.536124 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c437a1-81f6-4c50-8471-d80effd09c45 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:21.536401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c437a1-81f6-4c50-8471-d80effd09c45 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:21.536669 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c437a1-81f6-4c50-8471-d80effd09c45 None admin] PATH_INFO: `/v3/services/image` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:21.537241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c437a1-81f6-4c50-8471-d80effd09c45 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:21.538606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c437a1-81f6-4c50-8471-d80effd09c45 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c0c437a1-81f6-4c50-8471-d80effd09c45 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:21.547475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:21.548526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:21.549406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:21.549406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:21.549406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:21.549406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:21.549406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:21.549406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:21.549406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:21.549406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:21.549406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:21.549406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:21.549406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:21.549406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:21.549406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 14:40:21.549406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:21.551113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 497/999] 10.222.0.60 () {56 vars in 1105 bytes} [Mon Sep 19 14:40:21 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:21.554631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d564d3d-aa00-4a64-a0f4-4d04e9863e8a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:21.616344 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:21.670772 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d564d3d-aa00-4a64-a0f4-4d04e9863e8a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:21.710998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92f53e-8a38-4878-ab11-1a2bcde94784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:21.711617 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92f53e-8a38-4878-ab11-1a2bcde94784 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:21.711925 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92f53e-8a38-4878-ab11-1a2bcde94784 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:21.712201 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92f53e-8a38-4878-ab11-1a2bcde94784 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:21.712831 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92f53e-8a38-4878-ab11-1a2bcde94784 None admin] RBAC: Adding query filter params (name=image) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:21.713137 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92f53e-8a38-4878-ab11-1a2bcde94784 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:21.714496 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f92f53e-8a38-4878-ab11-1a2bcde94784 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:21.719734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 503/1000] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:21.725181 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c437a1-81f6-4c50-8471-d80effd09c45 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:21.790299 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:21.847163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c437a1-81f6-4c50-8471-d80effd09c45 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:21.887227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086121f-8e89-47e3-a49e-baf3c133e56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:21.889011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086121f-8e89-47e3-a49e-baf3c133e56f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:21.889439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086121f-8e89-47e3-a49e-baf3c133e56f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:21.889924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086121f-8e89-47e3-a49e-baf3c133e56f None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:21.890732 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086121f-8e89-47e3-a49e-baf3c133e56f None admin] RBAC: Adding query filter params (type=image) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:21.891177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086121f-8e89-47e3-a49e-baf3c133e56f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:21.892975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086121f-8e89-47e3-a49e-baf3c133e56f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:21.898379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 498/1001] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:21 2022] GET /identity/v3/services?type=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:21.961733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245bc16-a204-452a-8bcc-ac310a0faa14 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:21.961986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245bc16-a204-452a-8bcc-ac310a0faa14 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:21.962398 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245bc16-a204-452a-8bcc-ac310a0faa14 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:21.963850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 504/1002] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:21.970047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee64b5c-6f1f-42ae-af0c-da20ed524a23 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:21.970376 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee64b5c-6f1f-42ae-af0c-da20ed524a23 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:21.970606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee64b5c-6f1f-42ae-af0c-da20ed524a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:21.990758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0ee64b5c-6f1f-42ae-af0c-da20ed524a23 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:21.992041 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0ee64b5c-6f1f-42ae-af0c-da20ed524a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:22.078089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 499/1003] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:21 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:22.161015 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fd0ab-c50a-4cfa-9840-bb3b91cb7cbc None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:22.161531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fd0ab-c50a-4cfa-9840-bb3b91cb7cbc None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:22.161531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2fd0ab-c50a-4cfa-9840-bb3b91cb7cbc None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:22.162998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 505/1004] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:22.168056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb5284-11e6-4f12-8cfa-e0ffe7c6bf6a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:22.168347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb5284-11e6-4f12-8cfa-e0ffe7c6bf6a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:22.168624 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb5284-11e6-4f12-8cfa-e0ffe7c6bf6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:22.187401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-96fb5284-11e6-4f12-8cfa-e0ffe7c6bf6a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:22.188678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-96fb5284-11e6-4f12-8cfa-e0ffe7c6bf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:22.241720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:22.267229 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 500/1005] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:22 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:22.351760 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fd0ab-c50a-4cfa-9840-bb3b91cb7cbc None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:22.454848 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2fd0ab-c50a-4cfa-9840-bb3b91cb7cbc None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:22.468497 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:22.492635 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e538929-bd0d-47a4-b4cf-4333c1957c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:22.493489 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e538929-bd0d-47a4-b4cf-4333c1957c75 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:22.493736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e538929-bd0d-47a4-b4cf-4333c1957c75 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:22.494005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e538929-bd0d-47a4-b4cf-4333c1957c75 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:22.494577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e538929-bd0d-47a4-b4cf-4333c1957c75 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:22.495979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e538929-bd0d-47a4-b4cf-4333c1957c75 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-7e538929-bd0d-47a4-b4cf-4333c1957c75 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:22.505346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:22.506388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:22.507295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:22.507295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:22.507295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:22.507295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:22.507295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:22.507295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:22.507295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:22.507295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:22.507295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:22.507295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:22.507295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:22.507295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:22.507295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 19 14:40:22.507295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:22.511547 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 506/1006] 10.222.0.60 () {56 vars in 1126 bytes} [Mon Sep 19 14:40:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:22.512625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb5284-11e6-4f12-8cfa-e0ffe7c6bf6a None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:22.540085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e538929-bd0d-47a4-b4cf-4333c1957c75 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:22.619994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb5284-11e6-4f12-8cfa-e0ffe7c6bf6a None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:22.653755 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e538929-bd0d-47a4-b4cf-4333c1957c75 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:22.660368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cd9f1-7b83-41e2-961a-8ccff5da2625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:22.660983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cd9f1-7b83-41e2-961a-8ccff5da2625 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:22.661263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cd9f1-7b83-41e2-961a-8ccff5da2625 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:22.661990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cd9f1-7b83-41e2-961a-8ccff5da2625 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:22.662623 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63cd9f1-7b83-41e2-961a-8ccff5da2625 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:22.662922 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63cd9f1-7b83-41e2-961a-8ccff5da2625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:22.664696 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63cd9f1-7b83-41e2-961a-8ccff5da2625 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:22.677304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 501/1007] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Sep 19 14:40:22 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:22.678754 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cd9f1-7b83-41e2-961a-8ccff5da2625 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:22.694837 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95394c5a-f102-4a48-9fd2-49ed8c1fb5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:22.695865 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95394c5a-f102-4a48-9fd2-49ed8c1fb5b3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:22.696365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95394c5a-f102-4a48-9fd2-49ed8c1fb5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:22.696711 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95394c5a-f102-4a48-9fd2-49ed8c1fb5b3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:22.697294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95394c5a-f102-4a48-9fd2-49ed8c1fb5b3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:22.699357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95394c5a-f102-4a48-9fd2-49ed8c1fb5b3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-95394c5a-f102-4a48-9fd2-49ed8c1fb5b3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:22.711159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:22.712401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:22.713375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:22.713375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:22.713375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:22.713375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:22.713375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:22.713375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:22.713375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:22.713375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:22.713375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:22.713375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:22.713375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:22.713375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:22.713375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 19 14:40:22.713375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:22.713375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 507/1008] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:40:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:22.718951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95394c5a-f102-4a48-9fd2-49ed8c1fb5b3 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:22.732103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:22.766880 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:22.796217 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cd9f1-7b83-41e2-961a-8ccff5da2625 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:22.834775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95394c5a-f102-4a48-9fd2-49ed8c1fb5b3 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:22.835359 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a83ac-142a-4fcd-b6a0-d066604eac4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:22.836008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a83ac-142a-4fcd-b6a0-d066604eac4c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:22.836295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a83ac-142a-4fcd-b6a0-d066604eac4c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:22.836562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a83ac-142a-4fcd-b6a0-d066604eac4c None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:22.837182 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a83ac-142a-4fcd-b6a0-d066604eac4c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:22.837478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a83ac-142a-4fcd-b6a0-d066604eac4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:22.838840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a83ac-142a-4fcd-b6a0-d066604eac4c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:22.845204 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 502/1009] 10.222.0.60 () {56 vars in 1122 bytes} [Mon Sep 19 14:40:22 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:22.851614 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a83ac-142a-4fcd-b6a0-d066604eac4c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:22.876445 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff3fa7-df4f-465a-add6-14e144084791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:22.877056 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff3fa7-df4f-465a-add6-14e144084791 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:22.877337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff3fa7-df4f-465a-add6-14e144084791 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:22.877660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff3fa7-df4f-465a-add6-14e144084791 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:22.878299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff3fa7-df4f-465a-add6-14e144084791 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:22.878594 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff3fa7-df4f-465a-add6-14e144084791 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:22.880039 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff3fa7-df4f-465a-add6-14e144084791 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:22.891140 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 508/1010] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Sep 19 14:40:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:22.892393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff3fa7-df4f-465a-add6-14e144084791 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:22.961939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a83ac-142a-4fcd-b6a0-d066604eac4c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:22.969798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:22.997537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984421d-7fd2-4569-bcc1-e5d2661e5c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:22.998332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984421d-7fd2-4569-bcc1-e5d2661e5c2f None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:22.998923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984421d-7fd2-4569-bcc1-e5d2661e5c2f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:22.998923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984421d-7fd2-4569-bcc1-e5d2661e5c2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:22.999589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984421d-7fd2-4569-bcc1-e5d2661e5c2f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:23.000992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984421d-7fd2-4569-bcc1-e5d2661e5c2f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:23.004944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff3fa7-df4f-465a-add6-14e144084791 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:23.008213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:23.018986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 503/1011] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:40:22 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:40:23.065780 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acad12-a28b-4ba5-ac44-173612365a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:23.066397 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acad12-a28b-4ba5-ac44-173612365a44 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:23.066674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acad12-a28b-4ba5-ac44-173612365a44 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:23.066943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acad12-a28b-4ba5-ac44-173612365a44 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:23.067623 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acad12-a28b-4ba5-ac44-173612365a44 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:23.067996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acad12-a28b-4ba5-ac44-173612365a44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:23.069325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acad12-a28b-4ba5-ac44-173612365a44 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:23.078931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 509/1012] 10.222.0.60 () {56 vars in 1112 bytes} [Mon Sep 19 14:40:22 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:23.084556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984421d-7fd2-4569-bcc1-e5d2661e5c2f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:23.210044 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984421d-7fd2-4569-bcc1-e5d2661e5c2f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:23.246923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdef7fa-4312-4c8d-8241-c9d8324eea5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:23.248172 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdef7fa-4312-4c8d-8241-c9d8324eea5a None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:23.248454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdef7fa-4312-4c8d-8241-c9d8324eea5a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:23.248722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdef7fa-4312-4c8d-8241-c9d8324eea5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:23.249395 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdef7fa-4312-4c8d-8241-c9d8324eea5a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:23.250832 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdef7fa-4312-4c8d-8241-c9d8324eea5a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:23.271795 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 504/1013] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:40:23 2022] POST /identity/v3/endpoints => generated 334 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:40:24.121854 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a01ea-1a43-44c0-a3ac-4a073acf52c3 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:24.122486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a01ea-1a43-44c0-a3ac-4a073acf52c3 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:24.122486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a01ea-1a43-44c0-a3ac-4a073acf52c3 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:24.123971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 510/1014] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:24.128974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109226b-1c10-4436-a82a-69412993955e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:24.129417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109226b-1c10-4436-a82a-69412993955e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:24.129584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109226b-1c10-4436-a82a-69412993955e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:24.149720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1109226b-1c10-4436-a82a-69412993955e None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:24.151397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1109226b-1c10-4436-a82a-69412993955e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:24.179508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:24.241166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 505/1015] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:24 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:24.506567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a01ea-1a43-44c0-a3ac-4a073acf52c3 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:24.611965 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a01ea-1a43-44c0-a3ac-4a073acf52c3 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:24.649664 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab915849-cb7e-48d7-91d9-a6501279c920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:24.650272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab915849-cb7e-48d7-91d9-a6501279c920 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:24.650554 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab915849-cb7e-48d7-91d9-a6501279c920 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:24.650832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab915849-cb7e-48d7-91d9-a6501279c920 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:24.651490 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab915849-cb7e-48d7-91d9-a6501279c920 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:24.652883 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab915849-cb7e-48d7-91d9-a6501279c920 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:24.671928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 511/1016] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Sep 19 14:40:24 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:40:25.230570 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c35d67-1c38-4db3-b74e-4f1274c9bfb1 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:25.231090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c35d67-1c38-4db3-b74e-4f1274c9bfb1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:25.231090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c35d67-1c38-4db3-b74e-4f1274c9bfb1 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:25.233054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 506/1017] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:25.238778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1d114-c919-4e6d-a8bf-5aad1e12a860 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:25.239213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1d114-c919-4e6d-a8bf-5aad1e12a860 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:25.239383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac1d114-c919-4e6d-a8bf-5aad1e12a860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:25.257649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-aac1d114-c919-4e6d-a8bf-5aad1e12a860 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:25.258821 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-aac1d114-c919-4e6d-a8bf-5aad1e12a860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:25.277419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:25.336812 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 512/1018] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:25.559029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d1cc6-7519-44b5-8188-1811c19e2256 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:25.559401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d1cc6-7519-44b5-8188-1811c19e2256 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:25.560249 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d1cc6-7519-44b5-8188-1811c19e2256 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:25.561770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 507/1019] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:25.567512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df2b3-5901-4853-a083-572a89b3c554 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:25.567857 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df2b3-5901-4853-a083-572a89b3c554 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:25.568109 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5df2b3-5901-4853-a083-572a89b3c554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:25.587141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8c5df2b3-5901-4853-a083-572a89b3c554 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:25.588381 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8c5df2b3-5901-4853-a083-572a89b3c554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:25.607178 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d1cc6-7519-44b5-8188-1811c19e2256 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:25.669862 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 513/1020] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:25.714547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d1cc6-7519-44b5-8188-1811c19e2256 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:25.749912 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c2fc2-839a-4855-9072-5ade40072ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:25.750538 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c2fc2-839a-4855-9072-5ade40072ecb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:25.750820 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c2fc2-839a-4855-9072-5ade40072ecb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:25.751093 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c2fc2-839a-4855-9072-5ade40072ecb None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:25.752227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c2fc2-839a-4855-9072-5ade40072ecb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:25.753665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c2fc2-839a-4855-9072-5ade40072ecb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:25.755382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 508/1021] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:25.761095 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5df2b3-5901-4853-a083-572a89b3c554 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:25.842379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:25.862871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5df2b3-5901-4853-a083-572a89b3c554 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:25.898428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a40eb-3114-4f96-a1a9-ff0fda01d468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:25.899032 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a40eb-3114-4f96-a1a9-ff0fda01d468 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:25.899308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a40eb-3114-4f96-a1a9-ff0fda01d468 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:25.899576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a40eb-3114-4f96-a1a9-ff0fda01d468 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:25.900276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a40eb-3114-4f96-a1a9-ff0fda01d468 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:25.901697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a40eb-3114-4f96-a1a9-ff0fda01d468 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:25.929318 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 514/1022] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Sep 19 14:40:25 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:40:25.941372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c2fc2-839a-4855-9072-5ade40072ecb None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:25.962916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:26.060243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c2fc2-839a-4855-9072-5ade40072ecb None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:26.109027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21dec8-57dd-4973-bbee-7f5a0cddc839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:26.109909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21dec8-57dd-4973-bbee-7f5a0cddc839 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:26.110188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21dec8-57dd-4973-bbee-7f5a0cddc839 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:26.110457 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be21dec8-57dd-4973-bbee-7f5a0cddc839 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:26.121204 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be21dec8-57dd-4973-bbee-7f5a0cddc839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-be21dec8-57dd-4973-bbee-7f5a0cddc839 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:40:26.125961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:26.126923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:26.128174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:26.128174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:26.128174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:26.128174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:26.128174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:40:26.128174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:40:26.128174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:40:26.128174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:40:26.128174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:40:26.128174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:40:26.128174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 14:40:26.128174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:26.128174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 509/1023] 10.222.0.60 () {56 vars in 1120 bytes} [Mon Sep 19 14:40:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:26.133074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a40eb-3114-4f96-a1a9-ff0fda01d468 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:26.238282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a40eb-3114-4f96-a1a9-ff0fda01d468 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:26.267448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:26.272534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167033e-df92-41f4-ad7f-4e7d94d96181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:26.273166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167033e-df92-41f4-ad7f-4e7d94d96181 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:26.273439 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167033e-df92-41f4-ad7f-4e7d94d96181 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:26.273711 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8167033e-df92-41f4-ad7f-4e7d94d96181 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:26.274567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167033e-df92-41f4-ad7f-4e7d94d96181 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:26.274919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167033e-df92-41f4-ad7f-4e7d94d96181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:26.276858 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8167033e-df92-41f4-ad7f-4e7d94d96181 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:26.286265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 515/1024] 10.222.0.60 () {56 vars in 1115 bytes} [Mon Sep 19 14:40:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:26.291898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21dec8-57dd-4973-bbee-7f5a0cddc839 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:26.375285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:26.399767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be21dec8-57dd-4973-bbee-7f5a0cddc839 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:26.436997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d601c93-ac15-4d58-9438-9c42062c2266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:26.437631 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d601c93-ac15-4d58-9438-9c42062c2266 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:26.437910 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d601c93-ac15-4d58-9438-9c42062c2266 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:26.438181 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d601c93-ac15-4d58-9438-9c42062c2266 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:26.448366 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d601c93-ac15-4d58-9438-9c42062c2266 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:26.449817 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d601c93-ac15-4d58-9438-9c42062c2266 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-8d601c93-ac15-4d58-9438-9c42062c2266 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:26.464548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:26.465578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:26.467083 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:26.467083 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:26.467083 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:26.470953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 510/1025] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:26.471989 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167033e-df92-41f4-ad7f-4e7d94d96181 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:26.571953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8167033e-df92-41f4-ad7f-4e7d94d96181 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:26.615583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6946b7f-213e-460a-a7bf-be9779a2b42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:26.616252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6946b7f-213e-460a-a7bf-be9779a2b42a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:26.616536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6946b7f-213e-460a-a7bf-be9779a2b42a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:26.616804 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6946b7f-213e-460a-a7bf-be9779a2b42a None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:26.617434 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6946b7f-213e-460a-a7bf-be9779a2b42a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:26.617730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6946b7f-213e-460a-a7bf-be9779a2b42a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:26.619150 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6946b7f-213e-460a-a7bf-be9779a2b42a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:26.630945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 516/1026] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:26.637316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d601c93-ac15-4d58-9438-9c42062c2266 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:26.741231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d601c93-ac15-4d58-9438-9c42062c2266 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:26.771992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:26.783286 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f7a02-467a-4aec-9975-d0e8ea88bcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:26.784077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f7a02-467a-4aec-9975-d0e8ea88bcbd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:26.784429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f7a02-467a-4aec-9975-d0e8ea88bcbd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:26.784772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f7a02-467a-4aec-9975-d0e8ea88bcbd None admin] PATH_INFO: `/v3/users/nova` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:26.793391 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f7a02-467a-4aec-9975-d0e8ea88bcbd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:26.794968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659f7a02-467a-4aec-9975-d0e8ea88bcbd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-659f7a02-467a-4aec-9975-d0e8ea88bcbd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:40:26.806680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:26.808069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:26.809096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:26.809096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:26.809096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:40:26.809096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:40:26.809096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:40:26.809096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:40:26.809096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:40:26.809096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:40:26.809096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:40:26.809096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:26.811914 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 511/1027] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Sep 19 14:40:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:26.814381 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6946b7f-213e-460a-a7bf-be9779a2b42a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:26.874986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f678e22-9b36-4761-b35e-837d0dad523d None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:26.875279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f678e22-9b36-4761-b35e-837d0dad523d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:26.875557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f678e22-9b36-4761-b35e-837d0dad523d None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:26.877099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 512/1028] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:40:26.881831 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:26.882654 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ede2-a78c-4d0c-8429-2219b689069e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:26.882937 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ede2-a78c-4d0c-8429-2219b689069e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:26.883207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f6ede2-a78c-4d0c-8429-2219b689069e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:26.903165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-37f6ede2-a78c-4d0c-8429-2219b689069e None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:26.904918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-37f6ede2-a78c-4d0c-8429-2219b689069e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:26.938644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6946b7f-213e-460a-a7bf-be9779a2b42a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:26.973072 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb0522-44a3-4c60-a0fb-64a8656ed519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:26.973670 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb0522-44a3-4c60-a0fb-64a8656ed519 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:26.973944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb0522-44a3-4c60-a0fb-64a8656ed519 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:26.974217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb0522-44a3-4c60-a0fb-64a8656ed519 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:26.974918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb0522-44a3-4c60-a0fb-64a8656ed519 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:26.975218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb0522-44a3-4c60-a0fb-64a8656ed519 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:26.976157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 513/1029] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:40:26.976669 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb0522-44a3-4c60-a0fb-64a8656ed519 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:27.001212 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 517/1030] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Sep 19 14:40:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:27.003012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6ede2-a78c-4d0c-8429-2219b689069e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:27.111666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f6ede2-a78c-4d0c-8429-2219b689069e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:27.130194 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:27.148127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775383c-af02-4491-ae71-0ed30d1e1280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:27.148744 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775383c-af02-4491-ae71-0ed30d1e1280 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:27.149026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775383c-af02-4491-ae71-0ed30d1e1280 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:27.149301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775383c-af02-4491-ae71-0ed30d1e1280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:27.219488 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775383c-af02-4491-ae71-0ed30d1e1280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:27.220989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775383c-af02-4491-ae71-0ed30d1e1280 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:27.247178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb0522-44a3-4c60-a0fb-64a8656ed519 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:27.326716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 514/1031] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:40:27 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:40:27.332390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775383c-af02-4491-ae71-0ed30d1e1280 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:27.349818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:27.359884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb0522-44a3-4c60-a0fb-64a8656ed519 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:27.376784 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:27.401343 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c51b3-b4ad-46cb-abc3-a2d3ebfb2975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:27.401970 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c51b3-b4ad-46cb-abc3-a2d3ebfb2975 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:27.402230 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c51b3-b4ad-46cb-abc3-a2d3ebfb2975 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:27.402513 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c51b3-b4ad-46cb-abc3-a2d3ebfb2975 None admin] PATH_INFO: `/v3/services/image` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:27.403121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c51b3-b4ad-46cb-abc3-a2d3ebfb2975 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:27.404534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c51b3-b4ad-46cb-abc3-a2d3ebfb2975 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-661c51b3-b4ad-46cb-abc3-a2d3ebfb2975 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:27.413589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:27.414619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:27.415472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:27.415472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:27.415472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:27.415472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:27.415472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:27.415472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:27.415472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:27.415472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:27.415472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:27.415472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:27.415472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:27.415472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:27.415472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 14:40:27.415472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:27.415472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 518/1032] 10.222.0.60 () {56 vars in 1105 bytes} [Mon Sep 19 14:40:27 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:27.421314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c51b3-b4ad-46cb-abc3-a2d3ebfb2975 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:27.439201 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775383c-af02-4491-ae71-0ed30d1e1280 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:27.474888 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a40d20-b9bf-48c3-8757-d6cc85e0e6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:27.475510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a40d20-b9bf-48c3-8757-d6cc85e0e6e7 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:27.475785 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a40d20-b9bf-48c3-8757-d6cc85e0e6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:27.476238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a40d20-b9bf-48c3-8757-d6cc85e0e6e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:27.487247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a40d20-b9bf-48c3-8757-d6cc85e0e6e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:27.488721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a40d20-b9bf-48c3-8757-d6cc85e0e6e7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-e3a40d20-b9bf-48c3-8757-d6cc85e0e6e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:27.503231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:27.504402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:27.505630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:27.505630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:27.505630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:27.505630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 515/1033] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:27.509716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a40d20-b9bf-48c3-8757-d6cc85e0e6e7 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:27.531217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c51b3-b4ad-46cb-abc3-a2d3ebfb2975 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:27.572716 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b132f-dadd-4d17-8f6f-fcf58fc2488e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:27.573585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b132f-dadd-4d17-8f6f-fcf58fc2488e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:27.574341 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b132f-dadd-4d17-8f6f-fcf58fc2488e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:27.574624 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b132f-dadd-4d17-8f6f-fcf58fc2488e None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:27.576109 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b132f-dadd-4d17-8f6f-fcf58fc2488e None admin] RBAC: Adding query filter params (name=image) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:27.576417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b132f-dadd-4d17-8f6f-fcf58fc2488e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:27.578020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518b132f-dadd-4d17-8f6f-fcf58fc2488e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:27.588670 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 519/1034] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:27.588826 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:27.589800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b132f-dadd-4d17-8f6f-fcf58fc2488e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:27.623002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a40d20-b9bf-48c3-8757-d6cc85e0e6e7 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:27.641641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:27.657754 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324bad9-3c47-416a-ab0e-ef9ef645971a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:27.658261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324bad9-3c47-416a-ab0e-ef9ef645971a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:27.658548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324bad9-3c47-416a-ab0e-ef9ef645971a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:27.658827 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5324bad9-3c47-416a-ab0e-ef9ef645971a None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:27.659466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324bad9-3c47-416a-ab0e-ef9ef645971a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:27.659769 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324bad9-3c47-416a-ab0e-ef9ef645971a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:27.661171 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5324bad9-3c47-416a-ab0e-ef9ef645971a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:27.673643 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 516/1035] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:27.679348 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324bad9-3c47-416a-ab0e-ef9ef645971a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:27.708962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b132f-dadd-4d17-8f6f-fcf58fc2488e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:27.744921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e35e48-59cf-40b4-b0e8-2d77c554270b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:27.745553 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e35e48-59cf-40b4-b0e8-2d77c554270b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:27.746042 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e35e48-59cf-40b4-b0e8-2d77c554270b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:27.746156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e35e48-59cf-40b4-b0e8-2d77c554270b None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:27.746689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e35e48-59cf-40b4-b0e8-2d77c554270b None admin] RBAC: Adding query filter params (type=image) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:27.747078 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e35e48-59cf-40b4-b0e8-2d77c554270b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:27.748423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e35e48-59cf-40b4-b0e8-2d77c554270b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:27.753780 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 520/1036] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:27.759285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e35e48-59cf-40b4-b0e8-2d77c554270b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:27.788381 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5324bad9-3c47-416a-ab0e-ef9ef645971a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:27.821267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:27.829106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c452b8-a6f3-4300-92dd-046db9509e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:27.829741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c452b8-a6f3-4300-92dd-046db9509e26 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:27.830023 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c452b8-a6f3-4300-92dd-046db9509e26 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:27.830293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c452b8-a6f3-4300-92dd-046db9509e26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:27.837627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c452b8-a6f3-4300-92dd-046db9509e26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:27.839222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c452b8-a6f3-4300-92dd-046db9509e26 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:27.854025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-67c452b8-a6f3-4300-92dd-046db9509e26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:27.854025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:27.854025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:27.854025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:27.854025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:27.854025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:27.854025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:27.854025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:27.854025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:27.854025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:27.854025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:27.854025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:27.854025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:40:27.854025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:40:27.854025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:27.855008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:27.855959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:40:27.855959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:40:27.855959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:40:27.855959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:40:27.855959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:40:27.855959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:40:27.855959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:27.855959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:27.855959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 517/1037] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:40:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:27.861130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c452b8-a6f3-4300-92dd-046db9509e26 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:27.866821 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e35e48-59cf-40b4-b0e8-2d77c554270b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:27.869405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:27.909583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1043ba-2add-4e7c-afc1-3f105d1c7d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:27.910087 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1043ba-2add-4e7c-afc1-3f105d1c7d55 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:27.910365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1043ba-2add-4e7c-afc1-3f105d1c7d55 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:27.910637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1043ba-2add-4e7c-afc1-3f105d1c7d55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:27.911353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1043ba-2add-4e7c-afc1-3f105d1c7d55 None admin] RBAC: Adding query filter params (interface=public, service_id=a7c54cfd6ae4455ea2069ef6f55f6a75, region_id=RegionOne) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:27.911638 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1043ba-2add-4e7c-afc1-3f105d1c7d55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:27.913063 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1043ba-2add-4e7c-afc1-3f105d1c7d55 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:27.918697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 521/1038] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Sep 19 14:40:27 2022] GET /identity/v3/endpoints?service_id=a7c54cfd6ae4455ea2069ef6f55f6a75&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:27.924543 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1043ba-2add-4e7c-afc1-3f105d1c7d55 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:27.984786 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c452b8-a6f3-4300-92dd-046db9509e26 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:28.022409 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d6af6-ecf5-42d7-bbfb-12109f5d00e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:28.023050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d6af6-ecf5-42d7-bbfb-12109f5d00e1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:28.023349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d6af6-ecf5-42d7-bbfb-12109f5d00e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:28.023605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d6af6-ecf5-42d7-bbfb-12109f5d00e1 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:28.024354 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d6af6-ecf5-42d7-bbfb-12109f5d00e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:28.024659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d6af6-ecf5-42d7-bbfb-12109f5d00e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:28.026142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d6af6-ecf5-42d7-bbfb-12109f5d00e1 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:28.038582 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 518/1039] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:40:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:28.045035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d6af6-ecf5-42d7-bbfb-12109f5d00e1 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:28.053122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1043ba-2add-4e7c-afc1-3f105d1c7d55 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:28.097294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def065ba-09eb-4dbf-a128-815e8d1cd783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:28.097918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def065ba-09eb-4dbf-a128-815e8d1cd783 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:28.098194 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def065ba-09eb-4dbf-a128-815e8d1cd783 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:28.098459 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def065ba-09eb-4dbf-a128-815e8d1cd783 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:28.099121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def065ba-09eb-4dbf-a128-815e8d1cd783 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:28.100605 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def065ba-09eb-4dbf-a128-815e8d1cd783 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:28.109472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 522/1040] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Sep 19 14:40:27 2022] GET /identity/v3/services => generated 1957 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:40:28.115588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:28.177044 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d6af6-ecf5-42d7-bbfb-12109f5d00e1 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:28.201738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfc5b4-5cd6-41a3-9fdb-fb3584358a24 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:28.202031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfc5b4-5cd6-41a3-9fdb-fb3584358a24 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:28.202317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfc5b4-5cd6-41a3-9fdb-fb3584358a24 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:28.203917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 523/1041] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:40:28.209542 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf5b8c-2f02-40af-8de2-bc6ccd344bac None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:28.209837 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf5b8c-2f02-40af-8de2-bc6ccd344bac None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:28.210108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf5b8c-2f02-40af-8de2-bc6ccd344bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:28.221048 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715eae84-d511-42b6-bac4-112f411908a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:28.222514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715eae84-d511-42b6-bac4-112f411908a2 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:28.222859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715eae84-d511-42b6-bac4-112f411908a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:28.223198 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715eae84-d511-42b6-bac4-112f411908a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:28.224051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715eae84-d511-42b6-bac4-112f411908a2 None admin] RBAC: Adding query filter params (role.id=b88a5181c4564a8ca1acebcbd45465c0, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=59b5d2fd362b417c9b9be75f88a667ad) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:28.225012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715eae84-d511-42b6-bac4-112f411908a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:28.226443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715eae84-d511-42b6-bac4-112f411908a2 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:28.230322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5fdf5b8c-2f02-40af-8de2-bc6ccd344bac None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:28.232288 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5fdf5b8c-2f02-40af-8de2-bc6ccd344bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:28.233679 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 519/1042] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:40:28 2022] GET /identity/v3/role_assignments?user.id=59b5d2fd362b417c9b9be75f88a667ad&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=b88a5181c4564a8ca1acebcbd45465c0 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:28.266640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:28.325311 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 524/1043] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:40:28.608247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715eae84-d511-42b6-bac4-112f411908a2 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:28.754996 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715eae84-d511-42b6-bac4-112f411908a2 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:28.780431 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:28.817977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4020c-8f6f-4785-851c-cd90f41a7edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:28.819084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4020c-8f6f-4785-851c-cd90f41a7edd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:28.819598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4020c-8f6f-4785-851c-cd90f41a7edd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:28.820143 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4020c-8f6f-4785-851c-cd90f41a7edd None admin] PATH_INFO: `/v3/domains/default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:28.822117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4020c-8f6f-4785-851c-cd90f41a7edd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:28.824910 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e4020c-8f6f-4785-851c-cd90f41a7edd None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:28.832190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 520/1044] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:28.836706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf5b8c-2f02-40af-8de2-bc6ccd344bac None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:28.948238 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf5b8c-2f02-40af-8de2-bc6ccd344bac None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:28.989513 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4bea4-1d61-475c-a5cd-8c68e4461081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:28.990131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4bea4-1d61-475c-a5cd-8c68e4461081 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:28.990413 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4bea4-1d61-475c-a5cd-8c68e4461081 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:28.990681 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4bea4-1d61-475c-a5cd-8c68e4461081 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:28.991341 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4bea4-1d61-475c-a5cd-8c68e4461081 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:28.992801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4bea4-1d61-475c-a5cd-8c68e4461081 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:29.018497 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 525/1045] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:40:28 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:40:30.456876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1aac7-42c1-4925-8d66-6cb330d239d6 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:30.457478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1aac7-42c1-4925-8d66-6cb330d239d6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:30.457478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1aac7-42c1-4925-8d66-6cb330d239d6 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:30.458921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 521/1046] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:30.462884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09aa197-75c6-4eb0-bfc5-6a50193fc84d None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:30.463345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09aa197-75c6-4eb0-bfc5-6a50193fc84d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:30.463441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09aa197-75c6-4eb0-bfc5-6a50193fc84d None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:30.463771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27fa7e-4eb1-4983-bcb0-efebf0eab399 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:30.464090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27fa7e-4eb1-4983-bcb0-efebf0eab399 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:30.464355 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27fa7e-4eb1-4983-bcb0-efebf0eab399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:30.464907 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 526/1047] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:40:30.470233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33028a-37c4-4c7a-910e-e2f1f5b9a0f2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:30.470890 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33028a-37c4-4c7a-910e-e2f1f5b9a0f2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:30.471201 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33028a-37c4-4c7a-910e-e2f1f5b9a0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:30.484046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-7e27fa7e-4eb1-4983-bcb0-efebf0eab399 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:30.485235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7e27fa7e-4eb1-4983-bcb0-efebf0eab399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:30.492033 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-dd33028a-37c4-4c7a-910e-e2f1f5b9a0f2 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:30.493636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-dd33028a-37c4-4c7a-910e-e2f1f5b9a0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:30.509699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:30.556973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 522/1048] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:30.568879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 527/1049] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:40:30.829069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27fa7e-4eb1-4983-bcb0-efebf0eab399 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:30.854559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33028a-37c4-4c7a-910e-e2f1f5b9a0f2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:30.939109 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27fa7e-4eb1-4983-bcb0-efebf0eab399 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:30.952581 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:30.973285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33028a-37c4-4c7a-910e-e2f1f5b9a0f2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:30.974354 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c4de4-a30a-4d5b-a8ae-b489e8d75ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:30.975051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c4de4-a30a-4d5b-a8ae-b489e8d75ec7 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:30.975376 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c4de4-a30a-4d5b-a8ae-b489e8d75ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:30.975696 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c4de4-a30a-4d5b-a8ae-b489e8d75ec7 None admin] PATH_INFO: `/v3/services/image` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:30.976511 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c4de4-a30a-4d5b-a8ae-b489e8d75ec7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:30.980399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c4de4-a30a-4d5b-a8ae-b489e8d75ec7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-ce2c4de4-a30a-4d5b-a8ae-b489e8d75ec7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:30.993992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:30.995710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:30.997461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:30.997461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:30.997461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:30.997461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:30.997461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:30.997461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:30.997461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:30.997461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:30.997461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:30.997461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:30.997461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:40:30.997461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:40:30.997461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 19 14:40:30.997461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:30.997461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 523/1050] 10.222.0.60 () {56 vars in 1105 bytes} [Mon Sep 19 14:40:30 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:31.001595 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c4de4-a30a-4d5b-a8ae-b489e8d75ec7 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:31.021161 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b26dce-acdc-423a-928f-f1aecfde1110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:31.021794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b26dce-acdc-423a-928f-f1aecfde1110 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:31.022075 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b26dce-acdc-423a-928f-f1aecfde1110 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:31.022346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b26dce-acdc-423a-928f-f1aecfde1110 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:31.035247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b26dce-acdc-423a-928f-f1aecfde1110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-58b26dce-acdc-423a-928f-f1aecfde1110 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:40:31.040767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:31.041726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:31.042628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:31.042628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:31.042628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:31.042628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:31.042628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:40:31.042628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:40:31.042628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:40:31.042628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:40:31.042628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:40:31.042628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:40:31.042628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 14:40:31.042628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:31.046989 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 528/1051] 10.222.0.60 () {56 vars in 1120 bytes} [Mon Sep 19 14:40:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:31.047846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b26dce-acdc-423a-928f-f1aecfde1110 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:31.060421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:31.117775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c4de4-a30a-4d5b-a8ae-b489e8d75ec7 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:31.153193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ad4bf-620f-4d41-a10f-8b8fef6161ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:31.153820 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ad4bf-620f-4d41-a10f-8b8fef6161ec None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:31.154102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ad4bf-620f-4d41-a10f-8b8fef6161ec None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:31.154377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ad4bf-620f-4d41-a10f-8b8fef6161ec None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:31.154550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b26dce-acdc-423a-928f-f1aecfde1110 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:31.155014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ad4bf-620f-4d41-a10f-8b8fef6161ec None admin] RBAC: Adding query filter params (name=image) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:31.155324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ad4bf-620f-4d41-a10f-8b8fef6161ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:31.156868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ad4bf-620f-4d41-a10f-8b8fef6161ec None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:31.162524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 524/1052] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:31.168979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ad4bf-620f-4d41-a10f-8b8fef6161ec None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:31.197308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ff0f3-9be9-44f7-a246-24967f6797c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:31.197926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ff0f3-9be9-44f7-a246-24967f6797c5 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:31.198203 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ff0f3-9be9-44f7-a246-24967f6797c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:31.198472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ff0f3-9be9-44f7-a246-24967f6797c5 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:31.199124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ff0f3-9be9-44f7-a246-24967f6797c5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:31.199437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ff0f3-9be9-44f7-a246-24967f6797c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:31.201358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ff0f3-9be9-44f7-a246-24967f6797c5 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:31.209662 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 529/1053] 10.222.0.60 () {56 vars in 1115 bytes} [Mon Sep 19 14:40:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:31.215125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ff0f3-9be9-44f7-a246-24967f6797c5 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:31.217408 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:31.283648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ad4bf-620f-4d41-a10f-8b8fef6161ec None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:31.288289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:31.323703 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ff0f3-9be9-44f7-a246-24967f6797c5 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:31.323936 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ecaa8-b827-4870-9ae6-7ca1161a7c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:31.324440 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ecaa8-b827-4870-9ae6-7ca1161a7c46 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:31.324765 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ecaa8-b827-4870-9ae6-7ca1161a7c46 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:31.325663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ecaa8-b827-4870-9ae6-7ca1161a7c46 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:31.326380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ecaa8-b827-4870-9ae6-7ca1161a7c46 None admin] RBAC: Adding query filter params (type=image) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:31.326688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ecaa8-b827-4870-9ae6-7ca1161a7c46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:31.328091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ecaa8-b827-4870-9ae6-7ca1161a7c46 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:31.333783 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 525/1054] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:31.334943 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9aa3-7da8-4ed0-a4ba-576fc998c110 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:31.335230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9aa3-7da8-4ed0-a4ba-576fc998c110 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:31.335504 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9aa3-7da8-4ed0-a4ba-576fc998c110 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:31.339001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 526/1055] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:31.339987 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d9aa3-7da8-4ed0-a4ba-576fc998c110 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:31.366530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88447ff3-3862-4704-9c1f-5e773801af11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:31.367215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88447ff3-3862-4704-9c1f-5e773801af11 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:31.367589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88447ff3-3862-4704-9c1f-5e773801af11 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:31.368027 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88447ff3-3862-4704-9c1f-5e773801af11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:31.379888 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88447ff3-3862-4704-9c1f-5e773801af11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:31.381321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88447ff3-3862-4704-9c1f-5e773801af11 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-88447ff3-3862-4704-9c1f-5e773801af11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:31.396147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:31.402136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:31.403135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:31.403135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:31.403135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:31.403135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 530/1056] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:31.403135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c079114-a1d5-4134-836f-0e9bf25767ed None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:31.403135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c079114-a1d5-4134-836f-0e9bf25767ed None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:31.403135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c079114-a1d5-4134-836f-0e9bf25767ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:31.417259 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3c079114-a1d5-4134-836f-0e9bf25767ed None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:31.418461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3c079114-a1d5-4134-836f-0e9bf25767ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:31.446191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d9aa3-7da8-4ed0-a4ba-576fc998c110 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:31.448823 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:31.481253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcb04c-d5b2-41e1-9042-3bbc1e4f291f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:31.481938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcb04c-d5b2-41e1-9042-3bbc1e4f291f None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:31.482177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcb04c-d5b2-41e1-9042-3bbc1e4f291f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:31.482451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edcb04c-d5b2-41e1-9042-3bbc1e4f291f None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:31.483151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcb04c-d5b2-41e1-9042-3bbc1e4f291f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:31.484533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edcb04c-d5b2-41e1-9042-3bbc1e4f291f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:31.487619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 531/1057] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:31.488563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c079114-a1d5-4134-836f-0e9bf25767ed None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:31.505675 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 527/1058] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:40:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:40:31.553503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:31.609160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c079114-a1d5-4134-836f-0e9bf25767ed None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:31.655277 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c86760-81bc-4ffb-9cdc-604b44549e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:31.655971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c86760-81bc-4ffb-9cdc-604b44549e67 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:31.656231 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c86760-81bc-4ffb-9cdc-604b44549e67 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:31.656500 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c86760-81bc-4ffb-9cdc-604b44549e67 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:31.657162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c86760-81bc-4ffb-9cdc-604b44549e67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:31.657662 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c86760-81bc-4ffb-9cdc-604b44549e67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:31.659099 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c86760-81bc-4ffb-9cdc-604b44549e67 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:31.678537 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 532/1059] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:31.683535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcb04c-d5b2-41e1-9042-3bbc1e4f291f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:31.760410 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c86760-81bc-4ffb-9cdc-604b44549e67 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:31.809877 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edcb04c-d5b2-41e1-9042-3bbc1e4f291f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:31.847033 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ded7f7-77c6-4556-99c6-27f8fa1dd690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:31.847669 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ded7f7-77c6-4556-99c6-27f8fa1dd690 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:31.847979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ded7f7-77c6-4556-99c6-27f8fa1dd690 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:31.848252 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ded7f7-77c6-4556-99c6-27f8fa1dd690 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:31.856867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ded7f7-77c6-4556-99c6-27f8fa1dd690 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:31.858298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ded7f7-77c6-4556-99c6-27f8fa1dd690 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-75ded7f7-77c6-4556-99c6-27f8fa1dd690 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:40:31.869703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:31.875716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:31.876825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:31.876825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:31.876825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:40:31.876825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:40:31.876825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:40:31.876825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:40:31.876825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:40:31.876825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:40:31.876825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:40:31.876825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:31.876825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 528/1060] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Sep 19 14:40:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:31.877512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c86760-81bc-4ffb-9cdc-604b44549e67 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:31.877592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ded7f7-77c6-4556-99c6-27f8fa1dd690 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:31.889464 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:31.904330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:31.917301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac9003-bd59-4a84-adc4-9abb90dc2a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:31.917931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac9003-bd59-4a84-adc4-9abb90dc2a94 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:31.918212 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac9003-bd59-4a84-adc4-9abb90dc2a94 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:31.918480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac9003-bd59-4a84-adc4-9abb90dc2a94 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:31.930256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac9003-bd59-4a84-adc4-9abb90dc2a94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-c5ac9003-bd59-4a84-adc4-9abb90dc2a94 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:40:31.935073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:31.936094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:31.936958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:31.936958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:31.936958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:31.936958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:31.936958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:40:31.936958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:40:31.936958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:40:31.936958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:40:31.936958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:40:31.936958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:40:31.936958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:40:31.936958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:31.936958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 533/1061] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:40:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:31.942146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac9003-bd59-4a84-adc4-9abb90dc2a94 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:31.994124 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ded7f7-77c6-4556-99c6-27f8fa1dd690 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:32.035256 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca5dc8-8cf8-4cb2-a57a-9658b19f485c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:32.035944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca5dc8-8cf8-4cb2-a57a-9658b19f485c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:32.036216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca5dc8-8cf8-4cb2-a57a-9658b19f485c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:32.036913 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca5dc8-8cf8-4cb2-a57a-9658b19f485c None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:32.037343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca5dc8-8cf8-4cb2-a57a-9658b19f485c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:32.037692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca5dc8-8cf8-4cb2-a57a-9658b19f485c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:32.039373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca5dc8-8cf8-4cb2-a57a-9658b19f485c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:32.069325 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 529/1062] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Sep 19 14:40:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:32.074627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca5dc8-8cf8-4cb2-a57a-9658b19f485c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:32.080390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac9003-bd59-4a84-adc4-9abb90dc2a94 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:32.118776 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15d33c-4dcd-4582-a25a-beb2212f0f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:32.119394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15d33c-4dcd-4582-a25a-beb2212f0f76 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:32.119674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15d33c-4dcd-4582-a25a-beb2212f0f76 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:32.120035 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15d33c-4dcd-4582-a25a-beb2212f0f76 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:32.120647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15d33c-4dcd-4582-a25a-beb2212f0f76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:32.120950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15d33c-4dcd-4582-a25a-beb2212f0f76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:32.122415 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15d33c-4dcd-4582-a25a-beb2212f0f76 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:32.131664 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 534/1063] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:40:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:32.138895 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15d33c-4dcd-4582-a25a-beb2212f0f76 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:32.163347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:32.172217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:32.186632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca5dc8-8cf8-4cb2-a57a-9658b19f485c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:32.223752 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80bb41a-d994-4255-b0a7-24b0b2433ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:32.224428 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80bb41a-d994-4255-b0a7-24b0b2433ffe None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:32.224708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80bb41a-d994-4255-b0a7-24b0b2433ffe None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:32.225369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80bb41a-d994-4255-b0a7-24b0b2433ffe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:32.237397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80bb41a-d994-4255-b0a7-24b0b2433ffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:32.238860 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80bb41a-d994-4255-b0a7-24b0b2433ffe None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:32.250591 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15d33c-4dcd-4582-a25a-beb2212f0f76 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-f80bb41a-d994-4255-b0a7-24b0b2433ffe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:32.253844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:32.254805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:32.255744 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:32.255744 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:32.255744 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:32.255744 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 530/1064] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:32.260852 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80bb41a-d994-4255-b0a7-24b0b2433ffe None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:32.290441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe58dd5-f9cb-4150-8481-27773476b6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:32.291461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe58dd5-f9cb-4150-8481-27773476b6ff None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:32.291999 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe58dd5-f9cb-4150-8481-27773476b6ff None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:32.292335 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe58dd5-f9cb-4150-8481-27773476b6ff None admin] PATH_INFO: `/v3/users/f394a5a69993450e8bd6fc1e216c0ec7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:32.308354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe58dd5-f9cb-4150-8481-27773476b6ff None admin] RBAC: Authorizing `identity:get_user(user_id=f394a5a69993450e8bd6fc1e216c0ec7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:32.310757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe58dd5-f9cb-4150-8481-27773476b6ff None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:32.317309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 535/1065] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:40:32 2022] GET /identity/v3/users/f394a5a69993450e8bd6fc1e216c0ec7 => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:32.318419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe58dd5-f9cb-4150-8481-27773476b6ff None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:32.371690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80bb41a-d994-4255-b0a7-24b0b2433ffe None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:32.402516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:32.410502 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08355fa3-db3e-4069-8559-59c8d6bacb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:32.411122 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08355fa3-db3e-4069-8559-59c8d6bacb58 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:32.411410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08355fa3-db3e-4069-8559-59c8d6bacb58 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:32.411685 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08355fa3-db3e-4069-8559-59c8d6bacb58 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:32.412356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08355fa3-db3e-4069-8559-59c8d6bacb58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:32.412659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08355fa3-db3e-4069-8559-59c8d6bacb58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:32.414037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08355fa3-db3e-4069-8559-59c8d6bacb58 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:32.417731 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:32.428569 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 531/1066] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:32.436863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08355fa3-db3e-4069-8559-59c8d6bacb58 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:32.438560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe58dd5-f9cb-4150-8481-27773476b6ff None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:32.479282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fc867-e11c-4aa9-b570-5cf7527cfadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:32.480007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fc867-e11c-4aa9-b570-5cf7527cfadc None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:32.480336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fc867-e11c-4aa9-b570-5cf7527cfadc None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:32.480652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fc867-e11c-4aa9-b570-5cf7527cfadc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:32.553842 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fc867-e11c-4aa9-b570-5cf7527cfadc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:32.555291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fc867-e11c-4aa9-b570-5cf7527cfadc None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:32.556196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08355fa3-db3e-4069-8559-59c8d6bacb58 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:32.591816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76596875-78a5-4048-ab05-933c6d0e382a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:32.592782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76596875-78a5-4048-ab05-933c6d0e382a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:32.593268 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76596875-78a5-4048-ab05-933c6d0e382a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:32.593717 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76596875-78a5-4048-ab05-933c6d0e382a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:32.601001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76596875-78a5-4048-ab05-933c6d0e382a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:32.602749 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76596875-78a5-4048-ab05-933c6d0e382a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:32.615424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-76596875-78a5-4048-ab05-933c6d0e382a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:32.615424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:32.615424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:32.615424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:32.615424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:32.615424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:32.615424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:32.615424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:32.615424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:32.615424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:32.615424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:32.615424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:32.615424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:40:32.615424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:40:32.615424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:32.616554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:32.617678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:40:32.617678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:40:32.617678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:40:32.617678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:40:32.617678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:40:32.617678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:40:32.617678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:32.617678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:32.617678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 532/1067] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:40:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:32.622428 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76596875-78a5-4048-ab05-933c6d0e382a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:32.649600 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:32.670338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 536/1068] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:40:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:40:32.671255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fc867-e11c-4aa9-b570-5cf7527cfadc None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:32.680996 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:32.738584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76596875-78a5-4048-ab05-933c6d0e382a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:32.784773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad50cb-f31d-4e6e-9431-b707e7c75a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:32.785383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad50cb-f31d-4e6e-9431-b707e7c75a35 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:32.785665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad50cb-f31d-4e6e-9431-b707e7c75a35 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:32.785932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad50cb-f31d-4e6e-9431-b707e7c75a35 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:32.786612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad50cb-f31d-4e6e-9431-b707e7c75a35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:32.786914 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad50cb-f31d-4e6e-9431-b707e7c75a35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:32.788374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad50cb-f31d-4e6e-9431-b707e7c75a35 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:32.791151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fc867-e11c-4aa9-b570-5cf7527cfadc None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:32.800787 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 533/1069] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:40:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:32.806617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad50cb-f31d-4e6e-9431-b707e7c75a35 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:32.830267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff291d-45be-45cf-aeda-eaf1ea6241fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:32.830953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff291d-45be-45cf-aeda-eaf1ea6241fa None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:32.831238 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff291d-45be-45cf-aeda-eaf1ea6241fa None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:32.831508 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff291d-45be-45cf-aeda-eaf1ea6241fa None admin] PATH_INFO: `/v3/projects/19db855e38db4028b38cfe800b7454fa` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:32.832654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff291d-45be-45cf-aeda-eaf1ea6241fa None admin] RBAC: Authorizing `identity:get_project(project_id=19db855e38db4028b38cfe800b7454fa)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:32.834115 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff291d-45be-45cf-aeda-eaf1ea6241fa None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:32.835847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 537/1070] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:40:32 2022] GET /identity/v3/projects/19db855e38db4028b38cfe800b7454fa => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:32.842406 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff291d-45be-45cf-aeda-eaf1ea6241fa None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:32.907934 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:32.924611 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad50cb-f31d-4e6e-9431-b707e7c75a35 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:32.937189 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:32.952733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff291d-45be-45cf-aeda-eaf1ea6241fa None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:32.964345 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aeb844-f8f9-40e3-a85e-adfdece3707e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:32.965011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aeb844-f8f9-40e3-a85e-adfdece3707e None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:32.965294 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aeb844-f8f9-40e3-a85e-adfdece3707e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:32.966215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aeb844-f8f9-40e3-a85e-adfdece3707e None admin] PATH_INFO: `/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/59b5d2fd362b417c9b9be75f88a667ad/roles/b88a5181c4564a8ca1acebcbd45465c0` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:32.968714 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aeb844-f8f9-40e3-a85e-adfdece3707e None admin] RBAC: Authorizing `identity:create_grant(project_id=7dee98edf19d478ca4f746ff85579fba, user_id=59b5d2fd362b417c9b9be75f88a667ad, role_id=b88a5181c4564a8ca1acebcbd45465c0)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:32.970275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aeb844-f8f9-40e3-a85e-adfdece3707e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:32.993478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 534/1071] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:40:32 2022] PUT /identity/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/59b5d2fd362b417c9b9be75f88a667ad/roles/b88a5181c4564a8ca1acebcbd45465c0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:40:32.993729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df32d7c-2aef-44bc-bcd1-fdfef057454a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:32.993729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df32d7c-2aef-44bc-bcd1-fdfef057454a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:32.993729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df32d7c-2aef-44bc-bcd1-fdfef057454a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:32.993729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df32d7c-2aef-44bc-bcd1-fdfef057454a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:32.994073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df32d7c-2aef-44bc-bcd1-fdfef057454a None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=19db855e38db4028b38cfe800b7454fa, user.id=f394a5a69993450e8bd6fc1e216c0ec7) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:32.994073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df32d7c-2aef-44bc-bcd1-fdfef057454a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:32.994073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df32d7c-2aef-44bc-bcd1-fdfef057454a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:32.998738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 538/1072] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:40:32 2022] GET /identity/v3/role_assignments?user.id=f394a5a69993450e8bd6fc1e216c0ec7&scope.project.id=19db855e38db4028b38cfe800b7454fa&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:33.748036 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74354d4-625f-4955-a57b-c209fad7a1ce None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:33.748598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74354d4-625f-4955-a57b-c209fad7a1ce None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:33.748598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74354d4-625f-4955-a57b-c209fad7a1ce None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:33.750049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 535/1073] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:33.755044 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576c40f-3923-44c0-bea1-0e644b2c70ac None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:33.755484 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576c40f-3923-44c0-bea1-0e644b2c70ac None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:33.755660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576c40f-3923-44c0-bea1-0e644b2c70ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:33.774056 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-7576c40f-3923-44c0-bea1-0e644b2c70ac None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:33.775231 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7576c40f-3923-44c0-bea1-0e644b2c70ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:33.842628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 539/1074] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:34.107540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74354d4-625f-4955-a57b-c209fad7a1ce None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:34.217603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74354d4-625f-4955-a57b-c209fad7a1ce None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:34.256450 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb63264-a2a4-4b88-bce7-54d1286c1bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:34.257062 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb63264-a2a4-4b88-bce7-54d1286c1bb9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:34.257345 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb63264-a2a4-4b88-bce7-54d1286c1bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:34.257621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb63264-a2a4-4b88-bce7-54d1286c1bb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:34.298621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:34.330712 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb63264-a2a4-4b88-bce7-54d1286c1bb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:34.332239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb63264-a2a4-4b88-bce7-54d1286c1bb9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:34.431098 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 536/1075] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:40:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:40:34.439992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576c40f-3923-44c0-bea1-0e644b2c70ac None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:34.491392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:34.542923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576c40f-3923-44c0-bea1-0e644b2c70ac None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:34.578920 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a031546-623c-495b-b935-bc022c9f8617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:34.579548 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a031546-623c-495b-b935-bc022c9f8617 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:34.579836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a031546-623c-495b-b935-bc022c9f8617 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:34.580117 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a031546-623c-495b-b935-bc022c9f8617 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:34.590112 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a031546-623c-495b-b935-bc022c9f8617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:34.591583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a031546-623c-495b-b935-bc022c9f8617 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-1a031546-623c-495b-b935-bc022c9f8617 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:34.605966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:34.607487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:34.613808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:34.613808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:34.613808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:34.613808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 540/1076] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:34.614068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb63264-a2a4-4b88-bce7-54d1286c1bb9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:34.684480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:34.719401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb63264-a2a4-4b88-bce7-54d1286c1bb9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:34.753969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d4559d-73e7-40f3-a5a7-d8c561e8578e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:34.754635 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4559d-73e7-40f3-a5a7-d8c561e8578e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:34.754880 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4559d-73e7-40f3-a5a7-d8c561e8578e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:34.755103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d4559d-73e7-40f3-a5a7-d8c561e8578e None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:34.755735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4559d-73e7-40f3-a5a7-d8c561e8578e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:34.756057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4559d-73e7-40f3-a5a7-d8c561e8578e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:34.757414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d4559d-73e7-40f3-a5a7-d8c561e8578e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:34.773706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 537/1077] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:35.164638 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923f7eb-6479-4ee0-aa3d-8d3a0f8f99ed None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:35.165194 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923f7eb-6479-4ee0-aa3d-8d3a0f8f99ed None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:35.165194 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923f7eb-6479-4ee0-aa3d-8d3a0f8f99ed None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:35.166780 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 541/1078] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:40:35.172023 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef861a29-e3a3-4d78-9f5a-a7c148730604 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:35.172314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef861a29-e3a3-4d78-9f5a-a7c148730604 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:35.172587 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef861a29-e3a3-4d78-9f5a-a7c148730604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:35.176775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722a0fb-aa21-4eab-8387-6341eee79cb9 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:35.177057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722a0fb-aa21-4eab-8387-6341eee79cb9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:35.177322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722a0fb-aa21-4eab-8387-6341eee79cb9 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:35.178850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 542/1079] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:35.183884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164ae8c-9e3c-4601-899a-6cc1394fddf0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:35.184290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164ae8c-9e3c-4601-899a-6cc1394fddf0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:35.184647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164ae8c-9e3c-4601-899a-6cc1394fddf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:35.193514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ef861a29-e3a3-4d78-9f5a-a7c148730604 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:35.194718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ef861a29-e3a3-4d78-9f5a-a7c148730604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:35.205629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c164ae8c-9e3c-4601-899a-6cc1394fddf0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:35.206803 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c164ae8c-9e3c-4601-899a-6cc1394fddf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:35.267676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 538/1080] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:35.283282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 543/1081] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:35.532825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef861a29-e3a3-4d78-9f5a-a7c148730604 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:35.553695 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164ae8c-9e3c-4601-899a-6cc1394fddf0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:35.583135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:35.592704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:35.634100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef861a29-e3a3-4d78-9f5a-a7c148730604 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:35.654338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164ae8c-9e3c-4601-899a-6cc1394fddf0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:35.669234 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b928713-357f-495b-a4e9-f299e690d82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:35.669888 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b928713-357f-495b-a4e9-f299e690d82c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:35.670175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b928713-357f-495b-a4e9-f299e690d82c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:35.670448 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b928713-357f-495b-a4e9-f299e690d82c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:35.680932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b928713-357f-495b-a4e9-f299e690d82c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-9b928713-357f-495b-a4e9-f299e690d82c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:40:35.685784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:35.686923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:35.688035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:35.688035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:35.688035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:35.688035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:35.688035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:40:35.688035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:40:35.688035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:40:35.688035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:40:35.688035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:40:35.688035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:40:35.688035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:40:35.688035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:35.688035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 539/1082] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:40:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:35.692131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a31254-dc1b-4280-9763-dd56c6eae47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:35.692566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a31254-dc1b-4280-9763-dd56c6eae47c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:35.692844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a31254-dc1b-4280-9763-dd56c6eae47c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:35.692938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b928713-357f-495b-a4e9-f299e690d82c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:35.693118 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a31254-dc1b-4280-9763-dd56c6eae47c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:35.703520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a31254-dc1b-4280-9763-dd56c6eae47c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-84a31254-dc1b-4280-9763-dd56c6eae47c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:40:35.708301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:35.709523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:35.710392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:35.710392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:35.710392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:35.710392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:35.710392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:40:35.710392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:40:35.710392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:40:35.710392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:40:35.710392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:40:35.710392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:40:35.710392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 19 14:40:35.710392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:35.710392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 544/1083] 10.222.0.60 () {56 vars in 1120 bytes} [Mon Sep 19 14:40:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:35.715276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a31254-dc1b-4280-9763-dd56c6eae47c None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:35.790481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b928713-357f-495b-a4e9-f299e690d82c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:35.806298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:35.811034 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:35.816991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a31254-dc1b-4280-9763-dd56c6eae47c None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:35.825550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67da5e-27cc-418d-9baa-e89b13a76b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:35.826152 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67da5e-27cc-418d-9baa-e89b13a76b54 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:35.826432 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67da5e-27cc-418d-9baa-e89b13a76b54 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:35.826698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba67da5e-27cc-418d-9baa-e89b13a76b54 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:35.827354 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67da5e-27cc-418d-9baa-e89b13a76b54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:35.827650 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67da5e-27cc-418d-9baa-e89b13a76b54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:35.829062 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba67da5e-27cc-418d-9baa-e89b13a76b54 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:35.837613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 540/1084] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:40:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:35.843944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67da5e-27cc-418d-9baa-e89b13a76b54 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:35.851318 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7e206-ecb1-45f1-b8a7-944c3eb3113b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:35.851967 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7e206-ecb1-45f1-b8a7-944c3eb3113b None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:35.852252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7e206-ecb1-45f1-b8a7-944c3eb3113b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:35.852525 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db7e206-ecb1-45f1-b8a7-944c3eb3113b None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:35.853207 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7e206-ecb1-45f1-b8a7-944c3eb3113b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:35.853485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7e206-ecb1-45f1-b8a7-944c3eb3113b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:35.854846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db7e206-ecb1-45f1-b8a7-944c3eb3113b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:35.863319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 545/1085] 10.222.0.60 () {56 vars in 1115 bytes} [Mon Sep 19 14:40:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:35.869171 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7e206-ecb1-45f1-b8a7-944c3eb3113b None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:35.945149 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba67da5e-27cc-418d-9baa-e89b13a76b54 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:35.970472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db7e206-ecb1-45f1-b8a7-944c3eb3113b None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:35.977242 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15d784-e3b4-4bdc-8866-395fe7437463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:35.977846 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15d784-e3b4-4bdc-8866-395fe7437463 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:35.978125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15d784-e3b4-4bdc-8866-395fe7437463 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:35.978397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15d784-e3b4-4bdc-8866-395fe7437463 None admin] PATH_INFO: `/v3/users/f394a5a69993450e8bd6fc1e216c0ec7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:35.979526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15d784-e3b4-4bdc-8866-395fe7437463 None admin] RBAC: Authorizing `identity:get_user(user_id=f394a5a69993450e8bd6fc1e216c0ec7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:35.981025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15d784-e3b4-4bdc-8866-395fe7437463 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:35.983191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 541/1086] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:40:35 2022] GET /identity/v3/users/f394a5a69993450e8bd6fc1e216c0ec7 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:35.989489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15d784-e3b4-4bdc-8866-395fe7437463 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:36.015720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149707a2-3f0c-4450-ac67-3257636ce3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:36.016513 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149707a2-3f0c-4450-ac67-3257636ce3c6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:36.016937 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149707a2-3f0c-4450-ac67-3257636ce3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:36.017213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149707a2-3f0c-4450-ac67-3257636ce3c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:36.029829 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149707a2-3f0c-4450-ac67-3257636ce3c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:36.031285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149707a2-3f0c-4450-ac67-3257636ce3c6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:36.043304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-149707a2-3f0c-4450-ac67-3257636ce3c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:36.045962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:36.047128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:36.048431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:36.048431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:36.048431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:36.048431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 546/1087] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:36.053000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149707a2-3f0c-4450-ac67-3257636ce3c6 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:36.061837 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:36.100262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15d784-e3b4-4bdc-8866-395fe7437463 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:36.134275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360bedb-a779-47b2-887e-b823354178f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:36.134893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360bedb-a779-47b2-887e-b823354178f2 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:36.135169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360bedb-a779-47b2-887e-b823354178f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:36.135438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360bedb-a779-47b2-887e-b823354178f2 None admin] PATH_INFO: `/v3/projects/19db855e38db4028b38cfe800b7454fa` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:36.136682 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360bedb-a779-47b2-887e-b823354178f2 None admin] RBAC: Authorizing `identity:get_project(project_id=19db855e38db4028b38cfe800b7454fa)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:36.138703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360bedb-a779-47b2-887e-b823354178f2 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:36.140354 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 542/1088] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:40:35 2022] GET /identity/v3/projects/19db855e38db4028b38cfe800b7454fa => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:36.146145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360bedb-a779-47b2-887e-b823354178f2 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:36.159170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149707a2-3f0c-4450-ac67-3257636ce3c6 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:36.193424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60f99f-96a8-4b23-ab93-ce5cf720cc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:36.194073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60f99f-96a8-4b23-ab93-ce5cf720cc99 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:36.194291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60f99f-96a8-4b23-ab93-ce5cf720cc99 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:36.194573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60f99f-96a8-4b23-ab93-ce5cf720cc99 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:36.195175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60f99f-96a8-4b23-ab93-ce5cf720cc99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:36.195477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60f99f-96a8-4b23-ab93-ce5cf720cc99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:36.196850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60f99f-96a8-4b23-ab93-ce5cf720cc99 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:36.209527 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 547/1089] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:40:36.215069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60f99f-96a8-4b23-ab93-ce5cf720cc99 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:36.245310 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360bedb-a779-47b2-887e-b823354178f2 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:36.263080 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:36.279887 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ffa3d-6b20-402a-b6d4-8410da8e153b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:36.280551 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ffa3d-6b20-402a-b6d4-8410da8e153b None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:36.280690 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:36.280868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ffa3d-6b20-402a-b6d4-8410da8e153b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:36.281108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ffa3d-6b20-402a-b6d4-8410da8e153b None admin] PATH_INFO: `/v3/projects/19db855e38db4028b38cfe800b7454fa/users/f394a5a69993450e8bd6fc1e216c0ec7/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:36.283337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ffa3d-6b20-402a-b6d4-8410da8e153b None admin] RBAC: Authorizing `identity:create_grant(project_id=19db855e38db4028b38cfe800b7454fa, user_id=f394a5a69993450e8bd6fc1e216c0ec7, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:36.284807 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ffa3d-6b20-402a-b6d4-8410da8e153b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:36.298199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 543/1090] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:40:36 2022] PUT /identity/v3/projects/19db855e38db4028b38cfe800b7454fa/users/f394a5a69993450e8bd6fc1e216c0ec7/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:40:36.316938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60f99f-96a8-4b23-ab93-ce5cf720cc99 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:36.355894 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728955be-0a2a-4978-bda5-a3680866ffb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:36.356591 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728955be-0a2a-4978-bda5-a3680866ffb6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:36.356866 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728955be-0a2a-4978-bda5-a3680866ffb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:36.357136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728955be-0a2a-4978-bda5-a3680866ffb6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:36.365434 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728955be-0a2a-4978-bda5-a3680866ffb6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:36.367024 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728955be-0a2a-4978-bda5-a3680866ffb6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-728955be-0a2a-4978-bda5-a3680866ffb6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:40:36.380252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:36.381435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:36.382431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:36.382431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:36.382431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:40:36.382431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:40:36.382431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:40:36.382431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:40:36.382431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:40:36.382431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:40:36.382431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 19 14:40:36.382431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:36.382431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 548/1091] 10.222.0.60 () {56 vars in 1093 bytes} [Mon Sep 19 14:40:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:36.388183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ffa3d-6b20-402a-b6d4-8410da8e153b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:36.494850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ffa3d-6b20-402a-b6d4-8410da8e153b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:36.534295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d29a9-ddee-445c-bcf2-8385eb4fe48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:36.534902 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d29a9-ddee-445c-bcf2-8385eb4fe48d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:36.535185 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d29a9-ddee-445c-bcf2-8385eb4fe48d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:36.535461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d29a9-ddee-445c-bcf2-8385eb4fe48d None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:36.536217 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d29a9-ddee-445c-bcf2-8385eb4fe48d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:36.536524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d29a9-ddee-445c-bcf2-8385eb4fe48d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:36.537957 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226d29a9-ddee-445c-bcf2-8385eb4fe48d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:36.555616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 544/1092] 10.222.0.60 () {56 vars in 1133 bytes} [Mon Sep 19 14:40:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:36.561781 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728955be-0a2a-4978-bda5-a3680866ffb6 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:36.662975 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728955be-0a2a-4978-bda5-a3680866ffb6 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:36.688589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:36.697325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a92dbc-06ec-4161-aaab-d921ecfcdc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:36.697936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a92dbc-06ec-4161-aaab-d921ecfcdc1d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:36.698218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a92dbc-06ec-4161-aaab-d921ecfcdc1d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:36.698496 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a92dbc-06ec-4161-aaab-d921ecfcdc1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:36.767066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a92dbc-06ec-4161-aaab-d921ecfcdc1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:36.768530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a92dbc-06ec-4161-aaab-d921ecfcdc1d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:36.866608 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 549/1093] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:40:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:40:36.872843 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d29a9-ddee-445c-bcf2-8385eb4fe48d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:36.908102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:36.974609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226d29a9-ddee-445c-bcf2-8385eb4fe48d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:37.008344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c4378-0c6a-4428-8606-6a7afba58e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:37.008980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4378-0c6a-4428-8606-6a7afba58e05 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:37.009264 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4378-0c6a-4428-8606-6a7afba58e05 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:37.009545 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c4378-0c6a-4428-8606-6a7afba58e05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:37.020486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c4378-0c6a-4428-8606-6a7afba58e05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:37.021958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c4378-0c6a-4428-8606-6a7afba58e05 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-0e2c4378-0c6a-4428-8606-6a7afba58e05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:40:37.036566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:37.037953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:40:37.039315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:40:37.039315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:40:37.039315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:37.039315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 545/1094] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:37.043132 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a92dbc-06ec-4161-aaab-d921ecfcdc1d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:37.070377 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:37.143373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a92dbc-06ec-4161-aaab-d921ecfcdc1d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:37.179079 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad7357-1554-488b-86e7-6b8e0d8643b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:37.179676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad7357-1554-488b-86e7-6b8e0d8643b9 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:37.179989 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad7357-1554-488b-86e7-6b8e0d8643b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:37.180264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad7357-1554-488b-86e7-6b8e0d8643b9 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:37.180892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad7357-1554-488b-86e7-6b8e0d8643b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:37.181192 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad7357-1554-488b-86e7-6b8e0d8643b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:37.182536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad7357-1554-488b-86e7-6b8e0d8643b9 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:37.193989 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 550/1095] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:37.199644 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c4378-0c6a-4428-8606-6a7afba58e05 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:37.288829 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:37.301196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c4378-0c6a-4428-8606-6a7afba58e05 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:37.334597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91136d8f-b51a-40b0-ab87-452633611e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:37.335221 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91136d8f-b51a-40b0-ab87-452633611e43 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:37.335500 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91136d8f-b51a-40b0-ab87-452633611e43 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:37.335773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91136d8f-b51a-40b0-ab87-452633611e43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:37.342910 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91136d8f-b51a-40b0-ab87-452633611e43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:37.344392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91136d8f-b51a-40b0-ab87-452633611e43 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:37.355453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-91136d8f-b51a-40b0-ab87-452633611e43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:37.355453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:37.355453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:37.355453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:37.355453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:37.355453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:37.355453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:37.355453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:37.355453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:37.355453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:37.355453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:37.355453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:37.355453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:40:37.355453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:40:37.355453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:37.356608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:37.357563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:40:37.357563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:40:37.357563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:40:37.357563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:40:37.357563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:40:37.357563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:40:37.357563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:40:37.357563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:40:37.357563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 546/1096] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:40:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:40:37.362627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad7357-1554-488b-86e7-6b8e0d8643b9 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:37.446794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:37.462536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad7357-1554-488b-86e7-6b8e0d8643b9 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:37.495653 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f2682-dd14-4c89-9ebd-f2eef731bcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:37.496312 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f2682-dd14-4c89-9ebd-f2eef731bcf8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:37.496602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f2682-dd14-4c89-9ebd-f2eef731bcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:37.496905 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f2682-dd14-4c89-9ebd-f2eef731bcf8 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:37.497586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f2682-dd14-4c89-9ebd-f2eef731bcf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:37.497890 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f2682-dd14-4c89-9ebd-f2eef731bcf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:37.499298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f2682-dd14-4c89-9ebd-f2eef731bcf8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:37.512051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 551/1097] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:40:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:37.518707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91136d8f-b51a-40b0-ab87-452633611e43 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:37.619497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91136d8f-b51a-40b0-ab87-452633611e43 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:37.653824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3980575-46ca-491b-9dff-2cd94256643b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:37.654422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3980575-46ca-491b-9dff-2cd94256643b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:37.654700 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3980575-46ca-491b-9dff-2cd94256643b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:37.654976 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3980575-46ca-491b-9dff-2cd94256643b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:37.655678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3980575-46ca-491b-9dff-2cd94256643b None admin] RBAC: Adding query filter params (role.id=b88a5181c4564a8ca1acebcbd45465c0, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=59b5d2fd362b417c9b9be75f88a667ad) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:37.656021 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3980575-46ca-491b-9dff-2cd94256643b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:37.657439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3980575-46ca-491b-9dff-2cd94256643b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:37.663088 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 547/1098] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:40:37 2022] GET /identity/v3/role_assignments?user.id=59b5d2fd362b417c9b9be75f88a667ad&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=b88a5181c4564a8ca1acebcbd45465c0 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:38.433330 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e45aaa-3a27-4e64-8a09-58df85e4f06d None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:38.433858 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e45aaa-3a27-4e64-8a09-58df85e4f06d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:38.433858 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e45aaa-3a27-4e64-8a09-58df85e4f06d None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:38.435219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 552/1099] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:38.440643 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1e637-cd87-4b5a-822a-5826e956e3e6 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:38.441096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1e637-cd87-4b5a-822a-5826e956e3e6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:38.441191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1e637-cd87-4b5a-822a-5826e956e3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:38.459496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-02a1e637-cd87-4b5a-822a-5826e956e3e6 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:38.460761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-02a1e637-cd87-4b5a-822a-5826e956e3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:38.469160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:38.530117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 548/1100] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:38.804436 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e45aaa-3a27-4e64-8a09-58df85e4f06d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:38.911371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e45aaa-3a27-4e64-8a09-58df85e4f06d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:38.946418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fa8e8-07b0-4893-90dc-7b38ba5e093f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:38.947039 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fa8e8-07b0-4893-90dc-7b38ba5e093f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:38.947320 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fa8e8-07b0-4893-90dc-7b38ba5e093f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:38.947596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fa8e8-07b0-4893-90dc-7b38ba5e093f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:38.958542 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793fa8e8-07b0-4893-90dc-7b38ba5e093f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-793fa8e8-07b0-4893-90dc-7b38ba5e093f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:40:38.963272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:40:38.964407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:40:38.965454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:40:38.965454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:40:38.965454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:40:38.965454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:40:38.965454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:40:38.965454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:40:38.965454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:40:38.965454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:40:38.965454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:40:38.965454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:40:38.965454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:40:38.965454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:40:38.965454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 553/1101] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:40:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:40:38.971201 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1e637-cd87-4b5a-822a-5826e956e3e6 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:39.068082 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1e637-cd87-4b5a-822a-5826e956e3e6 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:39.101777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb771843-9bf9-42b8-ab4d-2b2a4fad7d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:39.102386 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb771843-9bf9-42b8-ab4d-2b2a4fad7d13 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:39.102662 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb771843-9bf9-42b8-ab4d-2b2a4fad7d13 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:39.102933 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb771843-9bf9-42b8-ab4d-2b2a4fad7d13 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:39.103585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb771843-9bf9-42b8-ab4d-2b2a4fad7d13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:39.103913 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb771843-9bf9-42b8-ab4d-2b2a4fad7d13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:39.105277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb771843-9bf9-42b8-ab4d-2b2a4fad7d13 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:39.113129 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 549/1102] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:40:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:39.119330 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fa8e8-07b0-4893-90dc-7b38ba5e093f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:39.129150 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:39.220952 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fa8e8-07b0-4893-90dc-7b38ba5e093f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:39.254697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77692c-8bed-4018-8f18-0d61e2f92dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:39.255319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77692c-8bed-4018-8f18-0d61e2f92dbe None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:39.255600 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77692c-8bed-4018-8f18-0d61e2f92dbe None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:39.255900 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a77692c-8bed-4018-8f18-0d61e2f92dbe None admin] PATH_INFO: `/v3/users/f394a5a69993450e8bd6fc1e216c0ec7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:39.257017 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77692c-8bed-4018-8f18-0d61e2f92dbe None admin] RBAC: Authorizing `identity:get_user(user_id=f394a5a69993450e8bd6fc1e216c0ec7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:39.258434 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a77692c-8bed-4018-8f18-0d61e2f92dbe None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:39.260017 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 554/1103] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:40:39 2022] GET /identity/v3/users/f394a5a69993450e8bd6fc1e216c0ec7 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:39.266734 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb771843-9bf9-42b8-ab4d-2b2a4fad7d13 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:39.270509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:39.365681 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb771843-9bf9-42b8-ab4d-2b2a4fad7d13 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:39.399034 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795995df-ad82-4b51-ae19-8c03908f11f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:39.399631 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795995df-ad82-4b51-ae19-8c03908f11f1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:39.399940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795995df-ad82-4b51-ae19-8c03908f11f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:39.400216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795995df-ad82-4b51-ae19-8c03908f11f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:39.464854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795995df-ad82-4b51-ae19-8c03908f11f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:39.466325 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795995df-ad82-4b51-ae19-8c03908f11f1 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:39.468940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:39.565322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 550/1104] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:40:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:40:39.571452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77692c-8bed-4018-8f18-0d61e2f92dbe None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:39.644511 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:39.669621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a77692c-8bed-4018-8f18-0d61e2f92dbe None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:39.702885 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87688c4-1638-4446-932b-45eea64f948c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:39.703508 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87688c4-1638-4446-932b-45eea64f948c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:39.703787 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87688c4-1638-4446-932b-45eea64f948c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:39.704087 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87688c4-1638-4446-932b-45eea64f948c None admin] PATH_INFO: `/v3/projects/19db855e38db4028b38cfe800b7454fa` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:39.705175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87688c4-1638-4446-932b-45eea64f948c None admin] RBAC: Authorizing `identity:get_project(project_id=19db855e38db4028b38cfe800b7454fa)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:39.706590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87688c4-1638-4446-932b-45eea64f948c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:39.708243 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 555/1105] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:40:39 2022] GET /identity/v3/projects/19db855e38db4028b38cfe800b7454fa => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:39.714407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795995df-ad82-4b51-ae19-8c03908f11f1 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:39.811153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795995df-ad82-4b51-ae19-8c03908f11f1 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:39.813489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:39.845233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc51b7-6087-48a3-9640-a618f0f8a36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:39.845836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc51b7-6087-48a3-9640-a618f0f8a36f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:39.846121 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc51b7-6087-48a3-9640-a618f0f8a36f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:39.846398 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc51b7-6087-48a3-9640-a618f0f8a36f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:39.847115 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc51b7-6087-48a3-9640-a618f0f8a36f None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=19db855e38db4028b38cfe800b7454fa, user.id=f394a5a69993450e8bd6fc1e216c0ec7) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:39.847424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc51b7-6087-48a3-9640-a618f0f8a36f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:39.848891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc51b7-6087-48a3-9640-a618f0f8a36f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:39.854561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 551/1106] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:40:39 2022] GET /identity/v3/role_assignments?user.id=f394a5a69993450e8bd6fc1e216c0ec7&scope.project.id=19db855e38db4028b38cfe800b7454fa&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:42.004830 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedd987-d64f-48f0-9fd1-1f955818a9c8 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:42.005362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedd987-d64f-48f0-9fd1-1f955818a9c8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:42.005362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedd987-d64f-48f0-9fd1-1f955818a9c8 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:42.006689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 556/1107] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:42.011911 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef20d7-e9b5-47ce-8114-af4f19db454e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:42.012322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef20d7-e9b5-47ce-8114-af4f19db454e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:42.012481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef20d7-e9b5-47ce-8114-af4f19db454e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:42.030412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1eef20d7-e9b5-47ce-8114-af4f19db454e None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:42.031582 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1eef20d7-e9b5-47ce-8114-af4f19db454e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:42.097680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 552/1108] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:40:42.366967 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedd987-d64f-48f0-9fd1-1f955818a9c8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:42.466709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedd987-d64f-48f0-9fd1-1f955818a9c8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:42.500343 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae1942-70b6-459c-af19-d803bb61794c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:42.500958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae1942-70b6-459c-af19-d803bb61794c None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:42.501232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae1942-70b6-459c-af19-d803bb61794c None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:42.501499 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae1942-70b6-459c-af19-d803bb61794c None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:42.502585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae1942-70b6-459c-af19-d803bb61794c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:42.504023 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ae1942-70b6-459c-af19-d803bb61794c None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:42.505544 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 557/1109] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:42.511295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef20d7-e9b5-47ce-8114-af4f19db454e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:42.602557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:42.607090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef20d7-e9b5-47ce-8114-af4f19db454e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:42.652306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb993c3-696a-4cfb-bb05-215844f3b516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:42.652955 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb993c3-696a-4cfb-bb05-215844f3b516 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:42.653236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb993c3-696a-4cfb-bb05-215844f3b516 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:42.653532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb993c3-696a-4cfb-bb05-215844f3b516 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:42.654219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb993c3-696a-4cfb-bb05-215844f3b516 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:42.655666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb993c3-696a-4cfb-bb05-215844f3b516 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:42.681001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 553/1110] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:40:42 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:40:44.922161 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0cdbe-d440-4128-859d-60dc07c29468 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:44.922711 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0cdbe-d440-4128-859d-60dc07c29468 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:44.922711 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e0cdbe-d440-4128-859d-60dc07c29468 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:44.924064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 558/1111] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:44.930218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7163f89-4a03-4951-9e4b-c64adc500df2 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:44.930737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7163f89-4a03-4951-9e4b-c64adc500df2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:44.930838 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7163f89-4a03-4951-9e4b-c64adc500df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:44.950330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e7163f89-4a03-4951-9e4b-c64adc500df2 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:44.951577 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e7163f89-4a03-4951-9e4b-c64adc500df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:45.022394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 554/1112] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:40:45.286767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0cdbe-d440-4128-859d-60dc07c29468 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:45.298791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:45.388178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e0cdbe-d440-4128-859d-60dc07c29468 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:45.420745 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526dc40-9714-4820-96c1-9cb92143ceb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:45.421365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526dc40-9714-4820-96c1-9cb92143ceb6 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:45.421645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526dc40-9714-4820-96c1-9cb92143ceb6 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:45.421921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526dc40-9714-4820-96c1-9cb92143ceb6 None admin] PATH_INFO: `/v3/roles/fe3bbad209634f16b1fcb66d6d49f665` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:45.423306 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526dc40-9714-4820-96c1-9cb92143ceb6 None admin] RBAC: Authorizing `identity:get_role(role_id=fe3bbad209634f16b1fcb66d6d49f665)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:45.424691 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526dc40-9714-4820-96c1-9cb92143ceb6 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:45.426256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 559/1113] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:40:45 2022] GET /identity/v3/roles/fe3bbad209634f16b1fcb66d6d49f665 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:45.432056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7163f89-4a03-4951-9e4b-c64adc500df2 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:45.515258 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:45.530933 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7163f89-4a03-4951-9e4b-c64adc500df2 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:45.564232 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f15e4-ef7a-471b-8de9-53fc02c05df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:45.564795 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f15e4-ef7a-471b-8de9-53fc02c05df3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:45.565074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f15e4-ef7a-471b-8de9-53fc02c05df3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:45.565347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f15e4-ef7a-471b-8de9-53fc02c05df3 None admin] PATH_INFO: `/v3/users/32b4c11381ee499882e8f93eb5be162e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:45.578621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f15e4-ef7a-471b-8de9-53fc02c05df3 None admin] RBAC: Authorizing `identity:get_user(user_id=32b4c11381ee499882e8f93eb5be162e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:45.580100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f15e4-ef7a-471b-8de9-53fc02c05df3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:45.581676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 555/1114] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:40:45 2022] GET /identity/v3/users/32b4c11381ee499882e8f93eb5be162e => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:45.587457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526dc40-9714-4820-96c1-9cb92143ceb6 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:45.666156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:45.688958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526dc40-9714-4820-96c1-9cb92143ceb6 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:45.721674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae000c53-7e5c-4bb6-82d3-606e37af0472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:45.722286 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae000c53-7e5c-4bb6-82d3-606e37af0472 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:45.722569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae000c53-7e5c-4bb6-82d3-606e37af0472 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:45.722843 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae000c53-7e5c-4bb6-82d3-606e37af0472 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:45.791855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae000c53-7e5c-4bb6-82d3-606e37af0472 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:45.793313 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae000c53-7e5c-4bb6-82d3-606e37af0472 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:45.868959 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:45.891994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 560/1115] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:40:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:40:45.897890 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f15e4-ef7a-471b-8de9-53fc02c05df3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:45.994238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f15e4-ef7a-471b-8de9-53fc02c05df3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:46.027677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd75901-0613-47cd-997b-eda03717e6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:46.028354 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd75901-0613-47cd-997b-eda03717e6d1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:46.028639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd75901-0613-47cd-997b-eda03717e6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:46.028914 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd75901-0613-47cd-997b-eda03717e6d1 None admin] PATH_INFO: `/v3/projects/19db855e38db4028b38cfe800b7454fa` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:46.030022 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd75901-0613-47cd-997b-eda03717e6d1 None admin] RBAC: Authorizing `identity:get_project(project_id=19db855e38db4028b38cfe800b7454fa)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:46.031456 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd75901-0613-47cd-997b-eda03717e6d1 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:46.033144 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 556/1116] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:40:45 2022] GET /identity/v3/projects/19db855e38db4028b38cfe800b7454fa => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:46.040109 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae000c53-7e5c-4bb6-82d3-606e37af0472 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:46.142066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae000c53-7e5c-4bb6-82d3-606e37af0472 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:46.174704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89f92d-ce8c-4517-bb0b-681147070ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:46.175301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89f92d-ce8c-4517-bb0b-681147070ed2 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:46.175581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89f92d-ce8c-4517-bb0b-681147070ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:46.176138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89f92d-ce8c-4517-bb0b-681147070ed2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:46.176852 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89f92d-ce8c-4517-bb0b-681147070ed2 None admin] RBAC: Adding query filter params (role.id=fe3bbad209634f16b1fcb66d6d49f665, scope.project.id=19db855e38db4028b38cfe800b7454fa, user.id=32b4c11381ee499882e8f93eb5be162e) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:46.177160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89f92d-ce8c-4517-bb0b-681147070ed2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:46.178577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89f92d-ce8c-4517-bb0b-681147070ed2 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:46.183765 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 561/1117] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:40:46 2022] GET /identity/v3/role_assignments?user.id=32b4c11381ee499882e8f93eb5be162e&scope.project.id=19db855e38db4028b38cfe800b7454fa&role.id=fe3bbad209634f16b1fcb66d6d49f665 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:48.346114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf256d55-336e-4957-80b1-543f447d5dd9 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:48.346618 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf256d55-336e-4957-80b1-543f447d5dd9 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:48.346618 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf256d55-336e-4957-80b1-543f447d5dd9 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:48.347909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 557/1118] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:48.353408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867f4b0-c7ef-456b-adc6-5aadc5480b6a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:48.353828 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867f4b0-c7ef-456b-adc6-5aadc5480b6a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:48.353943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867f4b0-c7ef-456b-adc6-5aadc5480b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:48.372440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e867f4b0-c7ef-456b-adc6-5aadc5480b6a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:48.373640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e867f4b0-c7ef-456b-adc6-5aadc5480b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:48.406916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:48.443282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 562/1119] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:40:48.710757 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf256d55-336e-4957-80b1-543f447d5dd9 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:48.723601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:48.814688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf256d55-336e-4957-80b1-543f447d5dd9 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:48.849385 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337b708-ff85-4fb9-a1da-5c8314e254ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:48.850011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337b708-ff85-4fb9-a1da-5c8314e254ca None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:48.850294 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337b708-ff85-4fb9-a1da-5c8314e254ca None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:48.850570 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337b708-ff85-4fb9-a1da-5c8314e254ca None admin] PATH_INFO: `/v3/roles/fe3bbad209634f16b1fcb66d6d49f665` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:48.851989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337b708-ff85-4fb9-a1da-5c8314e254ca None admin] RBAC: Authorizing `identity:get_role(role_id=fe3bbad209634f16b1fcb66d6d49f665)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:48.853367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337b708-ff85-4fb9-a1da-5c8314e254ca None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:48.854692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 558/1120] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:40:48 2022] GET /identity/v3/roles/fe3bbad209634f16b1fcb66d6d49f665 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:48.860709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867f4b0-c7ef-456b-adc6-5aadc5480b6a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:48.958672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867f4b0-c7ef-456b-adc6-5aadc5480b6a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:48.993389 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ce003-ee02-4855-9ea1-74fedc38b651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:48.993939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ce003-ee02-4855-9ea1-74fedc38b651 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:48.994216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ce003-ee02-4855-9ea1-74fedc38b651 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:48.994486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ce003-ee02-4855-9ea1-74fedc38b651 None admin] PATH_INFO: `/v3/users/32b4c11381ee499882e8f93eb5be162e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:48.995582 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ce003-ee02-4855-9ea1-74fedc38b651 None admin] RBAC: Authorizing `identity:get_user(user_id=32b4c11381ee499882e8f93eb5be162e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:48.997557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ce003-ee02-4855-9ea1-74fedc38b651 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:48.999108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 563/1121] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:40:48 2022] GET /identity/v3/users/32b4c11381ee499882e8f93eb5be162e => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:49.004949 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337b708-ff85-4fb9-a1da-5c8314e254ca None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:49.086508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:49.107975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337b708-ff85-4fb9-a1da-5c8314e254ca None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:49.141629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafa769-8884-47fc-8674-f994017fed81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:49.142330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafa769-8884-47fc-8674-f994017fed81 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:49.142614 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafa769-8884-47fc-8674-f994017fed81 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:49.142890 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafa769-8884-47fc-8674-f994017fed81 None admin] PATH_INFO: `/v3/projects/19db855e38db4028b38cfe800b7454fa` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:49.144025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fafa769-8884-47fc-8674-f994017fed81 None admin] RBAC: Authorizing `identity:get_project(project_id=19db855e38db4028b38cfe800b7454fa)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:49.145462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fafa769-8884-47fc-8674-f994017fed81 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:49.147079 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 559/1122] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:40:49 2022] GET /identity/v3/projects/19db855e38db4028b38cfe800b7454fa => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:49.153357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ce003-ee02-4855-9ea1-74fedc38b651 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:49.194580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:49.251474 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ce003-ee02-4855-9ea1-74fedc38b651 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:49.284869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979e84b-357d-4052-8b00-1ba47180552b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:49.285518 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979e84b-357d-4052-8b00-1ba47180552b None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:49.285795 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979e84b-357d-4052-8b00-1ba47180552b None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:49.286069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979e84b-357d-4052-8b00-1ba47180552b None admin] PATH_INFO: `/v3/projects/19db855e38db4028b38cfe800b7454fa/users/32b4c11381ee499882e8f93eb5be162e/roles/fe3bbad209634f16b1fcb66d6d49f665` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:49.288214 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979e84b-357d-4052-8b00-1ba47180552b None admin] RBAC: Authorizing `identity:create_grant(project_id=19db855e38db4028b38cfe800b7454fa, user_id=32b4c11381ee499882e8f93eb5be162e, role_id=fe3bbad209634f16b1fcb66d6d49f665)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:49.289657 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979e84b-357d-4052-8b00-1ba47180552b None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:49.304190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 564/1123] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:40:49 2022] PUT /identity/v3/projects/19db855e38db4028b38cfe800b7454fa/users/32b4c11381ee499882e8f93eb5be162e/roles/fe3bbad209634f16b1fcb66d6d49f665 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:40:51.422496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6f56f-359c-4483-b53a-6b4874b7ee7c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:51.423091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6f56f-359c-4483-b53a-6b4874b7ee7c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:51.423091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6f56f-359c-4483-b53a-6b4874b7ee7c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:51.424423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 560/1124] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:51.430071 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c10089-7b5a-4a86-99c7-d4754dcbb91d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:51.430455 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c10089-7b5a-4a86-99c7-d4754dcbb91d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:51.430611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c10089-7b5a-4a86-99c7-d4754dcbb91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:51.448934 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-46c10089-7b5a-4a86-99c7-d4754dcbb91d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:51.450734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-46c10089-7b5a-4a86-99c7-d4754dcbb91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:51.515737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 565/1125] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:51.783491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6f56f-359c-4483-b53a-6b4874b7ee7c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:51.885677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6f56f-359c-4483-b53a-6b4874b7ee7c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:51.921748 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321351f-4297-44c1-8331-540cc34b9168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:51.922378 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321351f-4297-44c1-8331-540cc34b9168 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:51.922657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321351f-4297-44c1-8331-540cc34b9168 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:51.922935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321351f-4297-44c1-8331-540cc34b9168 None admin] PATH_INFO: `/v3/roles/fe3bbad209634f16b1fcb66d6d49f665` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:51.924338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321351f-4297-44c1-8331-540cc34b9168 None admin] RBAC: Authorizing `identity:get_role(role_id=fe3bbad209634f16b1fcb66d6d49f665)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:51.925712 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321351f-4297-44c1-8331-540cc34b9168 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:51.927298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 561/1126] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:40:51 2022] GET /identity/v3/roles/fe3bbad209634f16b1fcb66d6d49f665 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:51.933252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c10089-7b5a-4a86-99c7-d4754dcbb91d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:51.969055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:52.042950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c10089-7b5a-4a86-99c7-d4754dcbb91d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:52.076955 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2994877-953d-4725-a644-aa92f6381ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:52.077586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2994877-953d-4725-a644-aa92f6381ca8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:52.077865 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2994877-953d-4725-a644-aa92f6381ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:52.078133 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2994877-953d-4725-a644-aa92f6381ca8 None admin] PATH_INFO: `/v3/users/32b4c11381ee499882e8f93eb5be162e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:52.079251 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2994877-953d-4725-a644-aa92f6381ca8 None admin] RBAC: Authorizing `identity:get_user(user_id=32b4c11381ee499882e8f93eb5be162e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:52.080700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2994877-953d-4725-a644-aa92f6381ca8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:52.082285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 566/1127] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:40:51 2022] GET /identity/v3/users/32b4c11381ee499882e8f93eb5be162e => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:52.088226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321351f-4297-44c1-8331-540cc34b9168 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:52.104252 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:52.190706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321351f-4297-44c1-8331-540cc34b9168 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:52.228426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6ee84-794f-4227-86f5-a16d6efa32ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:52.229169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6ee84-794f-4227-86f5-a16d6efa32ff None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:52.229469 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6ee84-794f-4227-86f5-a16d6efa32ff None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:52.230422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6ee84-794f-4227-86f5-a16d6efa32ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:52.300792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6ee84-794f-4227-86f5-a16d6efa32ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:52.302397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6ee84-794f-4227-86f5-a16d6efa32ff None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:52.317826 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:52.404492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 562/1128] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:40:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:40:52.410265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2994877-953d-4725-a644-aa92f6381ca8 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:52.505222 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:52.509181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2994877-953d-4725-a644-aa92f6381ca8 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:52.542084 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b5f91-a84f-41e3-bcd8-5e54fc528032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:52.542711 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b5f91-a84f-41e3-bcd8-5e54fc528032 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:52.542992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b5f91-a84f-41e3-bcd8-5e54fc528032 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:52.543264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b5f91-a84f-41e3-bcd8-5e54fc528032 None admin] PATH_INFO: `/v3/projects/19db855e38db4028b38cfe800b7454fa` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:52.544910 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b5f91-a84f-41e3-bcd8-5e54fc528032 None admin] RBAC: Authorizing `identity:get_project(project_id=19db855e38db4028b38cfe800b7454fa)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:52.546334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b5f91-a84f-41e3-bcd8-5e54fc528032 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:52.547959 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 567/1129] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:40:52 2022] GET /identity/v3/projects/19db855e38db4028b38cfe800b7454fa => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:52.554610 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6ee84-794f-4227-86f5-a16d6efa32ff None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:52.650593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6ee84-794f-4227-86f5-a16d6efa32ff None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:52.664826 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:52.686573 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89781b-e1b3-478d-8a88-5c5696b7e844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:52.687185 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89781b-e1b3-478d-8a88-5c5696b7e844 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:52.687470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89781b-e1b3-478d-8a88-5c5696b7e844 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:52.687764 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89781b-e1b3-478d-8a88-5c5696b7e844 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:52.688507 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89781b-e1b3-478d-8a88-5c5696b7e844 None admin] RBAC: Adding query filter params (role.id=fe3bbad209634f16b1fcb66d6d49f665, scope.project.id=19db855e38db4028b38cfe800b7454fa, user.id=32b4c11381ee499882e8f93eb5be162e) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:40:52.688815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89781b-e1b3-478d-8a88-5c5696b7e844 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:52.690338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89781b-e1b3-478d-8a88-5c5696b7e844 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:52.696869 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 563/1130] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:40:52 2022] GET /identity/v3/role_assignments?user.id=32b4c11381ee499882e8f93eb5be162e&scope.project.id=19db855e38db4028b38cfe800b7454fa&role.id=fe3bbad209634f16b1fcb66d6d49f665 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:54.946449 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d1084-2b41-4d70-8efa-ac5475add229 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:54.947009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d1084-2b41-4d70-8efa-ac5475add229 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:54.947009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d1084-2b41-4d70-8efa-ac5475add229 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:54.948370 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 568/1131] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:54.954210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b26b2-7488-40af-8d83-f67ad7511540 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:54.954652 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b26b2-7488-40af-8d83-f67ad7511540 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:54.954744 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b26b2-7488-40af-8d83-f67ad7511540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:54.972538 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0c6b26b2-7488-40af-8d83-f67ad7511540 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:54.973721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0c6b26b2-7488-40af-8d83-f67ad7511540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:55.041046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 564/1132] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:55.317884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d1084-2b41-4d70-8efa-ac5475add229 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:55.418632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d1084-2b41-4d70-8efa-ac5475add229 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:55.456652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0815ffa-c1d7-45cf-b895-ae23dfd874e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:55.457269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0815ffa-c1d7-45cf-b895-ae23dfd874e8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:55.457546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0815ffa-c1d7-45cf-b895-ae23dfd874e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:55.457817 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0815ffa-c1d7-45cf-b895-ae23dfd874e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:55.458899 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0815ffa-c1d7-45cf-b895-ae23dfd874e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:55.460359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0815ffa-c1d7-45cf-b895-ae23dfd874e8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:55.461921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 569/1133] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:55.468535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b26b2-7488-40af-8d83-f67ad7511540 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:55.562750 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b26b2-7488-40af-8d83-f67ad7511540 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:55.571583 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:55.596904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc69397-5336-46d2-862c-ce3575e8c655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:55.597498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc69397-5336-46d2-862c-ce3575e8c655 None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:55.597775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc69397-5336-46d2-862c-ce3575e8c655 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:55.598045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc69397-5336-46d2-862c-ce3575e8c655 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:55.598682 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc69397-5336-46d2-862c-ce3575e8c655 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:55.600135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc69397-5336-46d2-862c-ce3575e8c655 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:55.627057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 565/1134] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Sep 19 14:40:55 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:40:57.798423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8b2aa-7ec9-4ad1-ab9a-f3d9e4c41ded None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:57.798959 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8b2aa-7ec9-4ad1-ab9a-f3d9e4c41ded None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:57.798959 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f8b2aa-7ec9-4ad1-ab9a-f3d9e4c41ded None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:57.800332 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 570/1135] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:40:57.805892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4569da-2cb4-4ded-92f6-a71ca51030d0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:57.806314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4569da-2cb4-4ded-92f6-a71ca51030d0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:57.806458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4569da-2cb4-4ded-92f6-a71ca51030d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:57.825561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6f4569da-2cb4-4ded-92f6-a71ca51030d0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:40:57.826769 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6f4569da-2cb4-4ded-92f6-a71ca51030d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:40:57.894821 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 566/1136] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:40:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:40:58.164191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8b2aa-7ec9-4ad1-ab9a-f3d9e4c41ded None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:58.199060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:58.264392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f8b2aa-7ec9-4ad1-ab9a-f3d9e4c41ded None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:58.297465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640ada6-8ea8-4759-93a0-a9eef690e12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:58.298089 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640ada6-8ea8-4759-93a0-a9eef690e12a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:58.298370 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640ada6-8ea8-4759-93a0-a9eef690e12a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:58.298645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640ada6-8ea8-4759-93a0-a9eef690e12a None admin] PATH_INFO: `/v3/domains/default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:58.299745 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640ada6-8ea8-4759-93a0-a9eef690e12a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:58.301453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640ada6-8ea8-4759-93a0-a9eef690e12a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:58.302992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 571/1137] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:40:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:40:58.308704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4569da-2cb4-4ded-92f6-a71ca51030d0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:40:58.396781 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:40:58.407055 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4569da-2cb4-4ded-92f6-a71ca51030d0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:40:58.441345 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5c0c1-bc08-4d0f-9100-3a91e9b69a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:40:58.441950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5c0c1-bc08-4d0f-9100-3a91e9b69a3c None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:40:58.442232 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5c0c1-bc08-4d0f-9100-3a91e9b69a3c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:40:58.442506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5c0c1-bc08-4d0f-9100-3a91e9b69a3c None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:40:58.443160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5c0c1-bc08-4d0f-9100-3a91e9b69a3c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:40:58.444625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5c0c1-bc08-4d0f-9100-3a91e9b69a3c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:40:58.464942 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 567/1138] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:40:58 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:41:00.648187 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d026b-256f-4e51-a4a1-0b2e3164444f None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:00.648836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d026b-256f-4e51-a4a1-0b2e3164444f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:00.648836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d026b-256f-4e51-a4a1-0b2e3164444f None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:00.650053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 572/1139] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:41:00.655930 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bc659-8778-4d2c-8204-ecbc3ef80dd0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:00.656384 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bc659-8778-4d2c-8204-ecbc3ef80dd0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:00.656548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139bc659-8778-4d2c-8204-ecbc3ef80dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:00.675662 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-139bc659-8778-4d2c-8204-ecbc3ef80dd0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:41:00.677140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-139bc659-8778-4d2c-8204-ecbc3ef80dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:41:00.742705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 568/1140] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:41:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:41:01.009608 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d026b-256f-4e51-a4a1-0b2e3164444f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:01.111621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d026b-256f-4e51-a4a1-0b2e3164444f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:01.113158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:01.148172 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02f762-6206-435e-a6b5-5b8bc33f2a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:01.148794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02f762-6206-435e-a6b5-5b8bc33f2a14 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:01.149072 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02f762-6206-435e-a6b5-5b8bc33f2a14 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:01.149342 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02f762-6206-435e-a6b5-5b8bc33f2a14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:01.159442 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02f762-6206-435e-a6b5-5b8bc33f2a14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-9c02f762-6206-435e-a6b5-5b8bc33f2a14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:41:01.164146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:41:01.165338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:41:01.166503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:41:01.166503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:41:01.166503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:41:01.166503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:41:01.166503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:41:01.166503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:41:01.166503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:41:01.166503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:41:01.166503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:41:01.166503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:41:01.166503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:41:01.166503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:41:01.166503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 573/1141] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:41:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:41:01.171284 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bc659-8778-4d2c-8204-ecbc3ef80dd0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:01.245713 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:01.269585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139bc659-8778-4d2c-8204-ecbc3ef80dd0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:01.302470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70212bf-4d6e-430e-87c8-3616df7ec84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:01.303067 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70212bf-4d6e-430e-87c8-3616df7ec84f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:01.303355 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70212bf-4d6e-430e-87c8-3616df7ec84f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:01.303629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70212bf-4d6e-430e-87c8-3616df7ec84f None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:01.304309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70212bf-4d6e-430e-87c8-3616df7ec84f None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:41:01.304617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70212bf-4d6e-430e-87c8-3616df7ec84f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:01.305978 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70212bf-4d6e-430e-87c8-3616df7ec84f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:01.314523 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 569/1142] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:41:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:01.320477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02f762-6206-435e-a6b5-5b8bc33f2a14 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:01.416754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02f762-6206-435e-a6b5-5b8bc33f2a14 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:01.450913 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0cc27-fc84-476a-86fa-3f0112c679e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:01.451545 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0cc27-fc84-476a-86fa-3f0112c679e5 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:01.451911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0cc27-fc84-476a-86fa-3f0112c679e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:01.452202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0cc27-fc84-476a-86fa-3f0112c679e5 None admin] PATH_INFO: `/v3/users/9fad0242712e4437a16b862906fbc97e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:01.465874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0cc27-fc84-476a-86fa-3f0112c679e5 None admin] RBAC: Authorizing `identity:get_user(user_id=9fad0242712e4437a16b862906fbc97e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:01.467316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0cc27-fc84-476a-86fa-3f0112c679e5 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:01.468969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 574/1143] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:41:01 2022] GET /identity/v3/users/9fad0242712e4437a16b862906fbc97e => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:01.474839 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70212bf-4d6e-430e-87c8-3616df7ec84f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:01.576696 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70212bf-4d6e-430e-87c8-3616df7ec84f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:01.609439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710cdad-6d7e-4f7c-a52b-a6b841341578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:01.610042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710cdad-6d7e-4f7c-a52b-a6b841341578 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:01.610329 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710cdad-6d7e-4f7c-a52b-a6b841341578 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:01.610605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710cdad-6d7e-4f7c-a52b-a6b841341578 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:01.618817 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:01.678414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710cdad-6d7e-4f7c-a52b-a6b841341578 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:01.679889 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710cdad-6d7e-4f7c-a52b-a6b841341578 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:01.774572 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 570/1144] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:41:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:41:01.781153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0cc27-fc84-476a-86fa-3f0112c679e5 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:01.804737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:01.883041 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0cc27-fc84-476a-86fa-3f0112c679e5 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:01.916646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6592e05-fd83-4204-82e5-367da31a7a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:01.917270 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6592e05-fd83-4204-82e5-367da31a7a77 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:01.917557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6592e05-fd83-4204-82e5-367da31a7a77 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:01.917829 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6592e05-fd83-4204-82e5-367da31a7a77 None admin] PATH_INFO: `/v3/projects/2f504ecefa43456ea312c12a939de643` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:01.919600 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6592e05-fd83-4204-82e5-367da31a7a77 None admin] RBAC: Authorizing `identity:get_project(project_id=2f504ecefa43456ea312c12a939de643)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:01.921051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6592e05-fd83-4204-82e5-367da31a7a77 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:01.922649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 575/1145] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:41:01 2022] GET /identity/v3/projects/2f504ecefa43456ea312c12a939de643 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:01.929228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710cdad-6d7e-4f7c-a52b-a6b841341578 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:01.967599 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:02.026109 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710cdad-6d7e-4f7c-a52b-a6b841341578 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:02.059854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae71093-d351-413c-9d5c-6e77bd29ae87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:02.060502 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae71093-d351-413c-9d5c-6e77bd29ae87 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:02.060785 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae71093-d351-413c-9d5c-6e77bd29ae87 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:02.061061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae71093-d351-413c-9d5c-6e77bd29ae87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:02.061778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae71093-d351-413c-9d5c-6e77bd29ae87 None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=2f504ecefa43456ea312c12a939de643, user.id=9fad0242712e4437a16b862906fbc97e) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:41:02.062087 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae71093-d351-413c-9d5c-6e77bd29ae87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:02.063520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae71093-d351-413c-9d5c-6e77bd29ae87 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:02.068988 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 571/1146] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:41:01 2022] GET /identity/v3/role_assignments?user.id=9fad0242712e4437a16b862906fbc97e&scope.project.id=2f504ecefa43456ea312c12a939de643&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:04.239923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8485e5-2d20-4e4a-948b-06823a9fef2d None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:04.240494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8485e5-2d20-4e4a-948b-06823a9fef2d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:04.240494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8485e5-2d20-4e4a-948b-06823a9fef2d None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:04.241797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 576/1147] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:41:04.247588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7ac73-54c3-4255-a5d7-1f15cfcfc343 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:04.248050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7ac73-54c3-4255-a5d7-1f15cfcfc343 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:04.248174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7ac73-54c3-4255-a5d7-1f15cfcfc343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:04.266169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8fb7ac73-54c3-4255-a5d7-1f15cfcfc343 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:41:04.267361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8fb7ac73-54c3-4255-a5d7-1f15cfcfc343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:41:04.333066 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 572/1148] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:41:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:41:04.599415 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8485e5-2d20-4e4a-948b-06823a9fef2d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:04.693091 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:04.706284 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8485e5-2d20-4e4a-948b-06823a9fef2d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:04.740153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26203877-c85e-4f8f-97b4-b7ceb8f3ef71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:04.740860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26203877-c85e-4f8f-97b4-b7ceb8f3ef71 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:04.741057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26203877-c85e-4f8f-97b4-b7ceb8f3ef71 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:04.741329 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26203877-c85e-4f8f-97b4-b7ceb8f3ef71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:04.751168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26203877-c85e-4f8f-97b4-b7ceb8f3ef71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-26203877-c85e-4f8f-97b4-b7ceb8f3ef71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:41:04.755923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:41:04.757016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:41:04.757918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:41:04.757918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:41:04.757918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:41:04.757918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:41:04.757918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:41:04.757918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:41:04.757918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:41:04.757918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:41:04.757918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:41:04.757918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:41:04.757918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:41:04.757918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:41:04.757918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 577/1149] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:41:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:41:04.763055 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7ac73-54c3-4255-a5d7-1f15cfcfc343 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:04.792663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:04.860581 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7ac73-54c3-4255-a5d7-1f15cfcfc343 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:04.895178 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746286c-d952-4991-985d-60b641913c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:04.895783 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746286c-d952-4991-985d-60b641913c5d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:04.896086 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746286c-d952-4991-985d-60b641913c5d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:04.896359 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746286c-d952-4991-985d-60b641913c5d None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:04.897015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746286c-d952-4991-985d-60b641913c5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:41:04.897314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746286c-d952-4991-985d-60b641913c5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:04.898680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746286c-d952-4991-985d-60b641913c5d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:04.907130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 573/1150] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:41:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:04.912988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26203877-c85e-4f8f-97b4-b7ceb8f3ef71 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:05.012767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26203877-c85e-4f8f-97b4-b7ceb8f3ef71 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:05.045800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352136e4-557f-4dff-af5b-cdbd993a78a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:05.046424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352136e4-557f-4dff-af5b-cdbd993a78a7 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:05.046707 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352136e4-557f-4dff-af5b-cdbd993a78a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:05.046978 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352136e4-557f-4dff-af5b-cdbd993a78a7 None admin] PATH_INFO: `/v3/users/9fad0242712e4437a16b862906fbc97e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:05.048124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352136e4-557f-4dff-af5b-cdbd993a78a7 None admin] RBAC: Authorizing `identity:get_user(user_id=9fad0242712e4437a16b862906fbc97e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:05.049557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352136e4-557f-4dff-af5b-cdbd993a78a7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:05.051381 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 578/1151] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:41:04 2022] GET /identity/v3/users/9fad0242712e4437a16b862906fbc97e => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:05.057218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746286c-d952-4991-985d-60b641913c5d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:05.149616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:05.156714 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746286c-d952-4991-985d-60b641913c5d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:05.189668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b841f6-ffe5-45f2-89cd-b6ffece23a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:05.190279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b841f6-ffe5-45f2-89cd-b6ffece23a1c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:05.190555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b841f6-ffe5-45f2-89cd-b6ffece23a1c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:05.190823 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b841f6-ffe5-45f2-89cd-b6ffece23a1c None admin] PATH_INFO: `/v3/projects/2f504ecefa43456ea312c12a939de643` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:05.191932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b841f6-ffe5-45f2-89cd-b6ffece23a1c None admin] RBAC: Authorizing `identity:get_project(project_id=2f504ecefa43456ea312c12a939de643)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:05.193343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b841f6-ffe5-45f2-89cd-b6ffece23a1c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:05.194947 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 574/1152] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:41:05 2022] GET /identity/v3/projects/2f504ecefa43456ea312c12a939de643 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:05.200946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352136e4-557f-4dff-af5b-cdbd993a78a7 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:05.218381 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:05.297793 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352136e4-557f-4dff-af5b-cdbd993a78a7 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:05.331459 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cad607-a0cc-4da1-8484-a16aa1404bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:05.332152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cad607-a0cc-4da1-8484-a16aa1404bf1 None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:05.332435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cad607-a0cc-4da1-8484-a16aa1404bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:05.332709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cad607-a0cc-4da1-8484-a16aa1404bf1 None admin] PATH_INFO: `/v3/projects/2f504ecefa43456ea312c12a939de643/users/9fad0242712e4437a16b862906fbc97e/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:05.334606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cad607-a0cc-4da1-8484-a16aa1404bf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f504ecefa43456ea312c12a939de643, user_id=9fad0242712e4437a16b862906fbc97e, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:05.336059 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cad607-a0cc-4da1-8484-a16aa1404bf1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:05.349022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 579/1153] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:41:05 2022] PUT /identity/v3/projects/2f504ecefa43456ea312c12a939de643/users/9fad0242712e4437a16b862906fbc97e/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:41:07.453324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc0852-f9cc-480c-86b0-99dfff3d3ef9 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:07.453882 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc0852-f9cc-480c-86b0-99dfff3d3ef9 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:07.453882 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc0852-f9cc-480c-86b0-99dfff3d3ef9 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:07.455192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 575/1154] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:41:07.460517 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c10ce-b064-4d64-991c-622ff2ff8bbd None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:07.460953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c10ce-b064-4d64-991c-622ff2ff8bbd None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:07.461058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c10ce-b064-4d64-991c-622ff2ff8bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:07.478872 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b63c10ce-b064-4d64-991c-622ff2ff8bbd None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:41:07.480098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b63c10ce-b064-4d64-991c-622ff2ff8bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:41:07.544773 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 580/1155] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:41:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:41:07.813836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc0852-f9cc-480c-86b0-99dfff3d3ef9 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:07.912729 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc0852-f9cc-480c-86b0-99dfff3d3ef9 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:07.945746 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792377a8-0848-45be-9b1c-e1e9b67cf5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:07.946366 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792377a8-0848-45be-9b1c-e1e9b67cf5e6 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:07.946647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792377a8-0848-45be-9b1c-e1e9b67cf5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:07.946916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792377a8-0848-45be-9b1c-e1e9b67cf5e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:07.957227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792377a8-0848-45be-9b1c-e1e9b67cf5e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-792377a8-0848-45be-9b1c-e1e9b67cf5e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:41:07.961950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:41:07.962966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:41:07.964133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:41:07.964133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:41:07.964133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:41:07.964133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:41:07.964133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:41:07.964133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:41:07.964133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:41:07.964133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:41:07.964133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:41:07.964133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:41:07.964133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:41:07.964133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:41:07.964133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 576/1156] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:41:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:41:07.969760 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c10ce-b064-4d64-991c-622ff2ff8bbd None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:07.977725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:08.066617 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c10ce-b064-4d64-991c-622ff2ff8bbd None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:08.101483 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c553be-ecfc-42fd-8b1b-89f0c5613328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:08.102094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c553be-ecfc-42fd-8b1b-89f0c5613328 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:08.102375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c553be-ecfc-42fd-8b1b-89f0c5613328 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:08.102648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c553be-ecfc-42fd-8b1b-89f0c5613328 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:08.103301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c553be-ecfc-42fd-8b1b-89f0c5613328 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:41:08.103603 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c553be-ecfc-42fd-8b1b-89f0c5613328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:08.105013 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c553be-ecfc-42fd-8b1b-89f0c5613328 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:08.112928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 581/1157] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:41:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:08.119060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792377a8-0848-45be-9b1c-e1e9b67cf5e6 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:08.143318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:08.218293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792377a8-0848-45be-9b1c-e1e9b67cf5e6 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:08.250346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f855f7-13e3-4dba-a0d7-62748c1a8db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:08.250967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f855f7-13e3-4dba-a0d7-62748c1a8db3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:08.251250 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f855f7-13e3-4dba-a0d7-62748c1a8db3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:08.251528 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f855f7-13e3-4dba-a0d7-62748c1a8db3 None admin] PATH_INFO: `/v3/users/9fad0242712e4437a16b862906fbc97e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:08.252673 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f855f7-13e3-4dba-a0d7-62748c1a8db3 None admin] RBAC: Authorizing `identity:get_user(user_id=9fad0242712e4437a16b862906fbc97e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:08.254108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f855f7-13e3-4dba-a0d7-62748c1a8db3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:08.255673 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 577/1158] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:41:08 2022] GET /identity/v3/users/9fad0242712e4437a16b862906fbc97e => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:08.261478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c553be-ecfc-42fd-8b1b-89f0c5613328 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:08.327279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:08.360608 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c553be-ecfc-42fd-8b1b-89f0c5613328 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:08.393638 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ae138-5055-42c0-b899-e2c6bef95b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:08.394229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ae138-5055-42c0-b899-e2c6bef95b4a None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:08.394504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ae138-5055-42c0-b899-e2c6bef95b4a None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:08.394771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ae138-5055-42c0-b899-e2c6bef95b4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:08.458997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ae138-5055-42c0-b899-e2c6bef95b4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:08.460466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ae138-5055-42c0-b899-e2c6bef95b4a None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:08.520688 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:08.554896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 582/1159] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:41:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:41:08.560807 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f855f7-13e3-4dba-a0d7-62748c1a8db3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:08.648031 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:08.658501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f855f7-13e3-4dba-a0d7-62748c1a8db3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:08.691687 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d7cce-4dd7-4471-9391-861712137e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:08.692336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d7cce-4dd7-4471-9391-861712137e03 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:08.692621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d7cce-4dd7-4471-9391-861712137e03 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:08.692897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d7cce-4dd7-4471-9391-861712137e03 None admin] PATH_INFO: `/v3/projects/2f504ecefa43456ea312c12a939de643` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:08.694007 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d7cce-4dd7-4471-9391-861712137e03 None admin] RBAC: Authorizing `identity:get_project(project_id=2f504ecefa43456ea312c12a939de643)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:08.695443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d7cce-4dd7-4471-9391-861712137e03 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:08.697097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 578/1160] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:41:08 2022] GET /identity/v3/projects/2f504ecefa43456ea312c12a939de643 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:08.703373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ae138-5055-42c0-b899-e2c6bef95b4a None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:08.800102 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ae138-5055-42c0-b899-e2c6bef95b4a None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:08.833465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76761a8-5867-4106-a93c-142256be4b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:08.834057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76761a8-5867-4106-a93c-142256be4b1d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:08.834332 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76761a8-5867-4106-a93c-142256be4b1d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:08.834600 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76761a8-5867-4106-a93c-142256be4b1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:08.835303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76761a8-5867-4106-a93c-142256be4b1d None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=2f504ecefa43456ea312c12a939de643, user.id=9fad0242712e4437a16b862906fbc97e) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:41:08.835605 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76761a8-5867-4106-a93c-142256be4b1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:08.837050 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76761a8-5867-4106-a93c-142256be4b1d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:08.843093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 583/1161] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:41:08 2022] GET /identity/v3/role_assignments?user.id=9fad0242712e4437a16b862906fbc97e&scope.project.id=2f504ecefa43456ea312c12a939de643&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:11.026504 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eaaff4-f3b8-4653-bc42-acb9c95aef03 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:11.027045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eaaff4-f3b8-4653-bc42-acb9c95aef03 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:11.027045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eaaff4-f3b8-4653-bc42-acb9c95aef03 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:11.028392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 579/1162] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:41:11.033666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de9669-1696-4dea-bd80-0820ab01454b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:11.034083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de9669-1696-4dea-bd80-0820ab01454b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:11.034215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de9669-1696-4dea-bd80-0820ab01454b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:11.053953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-82de9669-1696-4dea-bd80-0820ab01454b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:41:11.055150 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-82de9669-1696-4dea-bd80-0820ab01454b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:41:11.081440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:11.123705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 584/1163] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:41:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:41:11.392178 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eaaff4-f3b8-4653-bc42-acb9c95aef03 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:11.491844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eaaff4-f3b8-4653-bc42-acb9c95aef03 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:11.523674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fa1ff-5b0f-4dc8-8cf9-bbcc9d33ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:11.524299 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fa1ff-5b0f-4dc8-8cf9-bbcc9d33ec09 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:11.524580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fa1ff-5b0f-4dc8-8cf9-bbcc9d33ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:11.524853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892fa1ff-5b0f-4dc8-8cf9-bbcc9d33ec09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:11.551089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:11.591223 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fa1ff-5b0f-4dc8-8cf9-bbcc9d33ec09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:11.592679 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892fa1ff-5b0f-4dc8-8cf9-bbcc9d33ec09 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:11.688667 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 580/1164] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:41:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:41:11.694611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de9669-1696-4dea-bd80-0820ab01454b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:11.790771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de9669-1696-4dea-bd80-0820ab01454b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:11.823378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2626e-e1e0-465d-ae7e-097445f8bc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:11.824023 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2626e-e1e0-465d-ae7e-097445f8bc88 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:11.824791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2626e-e1e0-465d-ae7e-097445f8bc88 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:11.825065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2626e-e1e0-465d-ae7e-097445f8bc88 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:11.831709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2626e-e1e0-465d-ae7e-097445f8bc88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:11.833176 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e2626e-e1e0-465d-ae7e-097445f8bc88 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-01e2626e-e1e0-465d-ae7e-097445f8bc88 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:41:11.844390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:41:11.845599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:41:11.846706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:41:11.846706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:41:11.846706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:41:11.846706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:41:11.846706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:41:11.846706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:41:11.846706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:41:11.846706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:41:11.846706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:41:11.846706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:41:11.846706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 19 14:41:11.846706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:41:11.846706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 19 14:41:11.846706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:41:11.846706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 585/1165] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Sep 19 14:41:11 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:41:11.851656 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fa1ff-5b0f-4dc8-8cf9-bbcc9d33ec09 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:11.910711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:11.952123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892fa1ff-5b0f-4dc8-8cf9-bbcc9d33ec09 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:11.985145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d285191-fa35-4d15-9768-2a15efcd9b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:11.985745 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d285191-fa35-4d15-9768-2a15efcd9b0d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:11.986026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d285191-fa35-4d15-9768-2a15efcd9b0d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:11.986300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d285191-fa35-4d15-9768-2a15efcd9b0d None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:11.986936 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d285191-fa35-4d15-9768-2a15efcd9b0d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:41:11.987240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d285191-fa35-4d15-9768-2a15efcd9b0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:11.988639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d285191-fa35-4d15-9768-2a15efcd9b0d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:11.995702 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 581/1166] 10.222.0.60 () {56 vars in 1115 bytes} [Mon Sep 19 14:41:11 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:14.116071 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6e7bf-f39b-41c6-82cb-afd69642135a None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:14.116586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6e7bf-f39b-41c6-82cb-afd69642135a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:14.116691 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe6e7bf-f39b-41c6-82cb-afd69642135a None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:14.117970 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 586/1167] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:41:14.123134 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f7a71-b816-4da5-a109-2c84a982b0a2 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:14.123551 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f7a71-b816-4da5-a109-2c84a982b0a2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:14.123691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f7a71-b816-4da5-a109-2c84a982b0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:14.142266 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-da4f7a71-b816-4da5-a109-2c84a982b0a2 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:41:14.143465 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-da4f7a71-b816-4da5-a109-2c84a982b0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:41:14.210508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 582/1168] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:41:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:41:14.478241 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6e7bf-f39b-41c6-82cb-afd69642135a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:14.502921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:14.578845 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe6e7bf-f39b-41c6-82cb-afd69642135a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:14.613879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b57c0-579d-4175-b719-413bcd7aa6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:14.614503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b57c0-579d-4175-b719-413bcd7aa6de None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:14.614786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b57c0-579d-4175-b719-413bcd7aa6de None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:14.615055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b57c0-579d-4175-b719-413bcd7aa6de None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:14.615777 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b57c0-579d-4175-b719-413bcd7aa6de None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:14.617217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b57c0-579d-4175-b719-413bcd7aa6de None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:14.633011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 587/1169] 10.222.0.60 () {60 vars in 1138 bytes} [Mon Sep 19 14:41:14 2022] POST /identity/v3/domains => generated 267 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:41:16.776423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520adf5-bb7d-4aa9-903a-28c810e65d31 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:16.776988 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520adf5-bb7d-4aa9-903a-28c810e65d31 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:16.776988 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520adf5-bb7d-4aa9-903a-28c810e65d31 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:16.778793 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 583/1170] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:41:16.784218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025a668-a5eb-4e88-bf02-f8be69a4f7e1 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:16.784651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025a668-a5eb-4e88-bf02-f8be69a4f7e1 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:16.784769 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025a668-a5eb-4e88-bf02-f8be69a4f7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:16.803321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e025a668-a5eb-4e88-bf02-f8be69a4f7e1 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:41:16.804536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e025a668-a5eb-4e88-bf02-f8be69a4f7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:41:16.870846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 588/1171] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:41:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:41:17.142816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520adf5-bb7d-4aa9-903a-28c810e65d31 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:17.201283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:17.250597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520adf5-bb7d-4aa9-903a-28c810e65d31 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:17.288016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3debf3d3-81d1-48c7-8203-c652f8b1a8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:17.288648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debf3d3-81d1-48c7-8203-c652f8b1a8ea None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:17.288932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debf3d3-81d1-48c7-8203-c652f8b1a8ea None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:17.289206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3debf3d3-81d1-48c7-8203-c652f8b1a8ea None admin] PATH_INFO: `/v3/domains/fe3e6215cdba41c6b85be13a81e263d9` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:17.299477 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debf3d3-81d1-48c7-8203-c652f8b1a8ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=fe3e6215cdba41c6b85be13a81e263d9)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:17.300981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3debf3d3-81d1-48c7-8203-c652f8b1a8ea None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:17.302839 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 584/1172] 10.222.0.60 () {56 vars in 1183 bytes} [Mon Sep 19 14:41:17 2022] GET /identity/v3/domains/fe3e6215cdba41c6b85be13a81e263d9 => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:17.308768 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025a668-a5eb-4e88-bf02-f8be69a4f7e1 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:17.322405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:17.409027 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025a668-a5eb-4e88-bf02-f8be69a4f7e1 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:17.444411 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7582f45-cf6d-42d3-b67e-c83a72e25727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:17.445017 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7582f45-cf6d-42d3-b67e-c83a72e25727 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:17.445299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7582f45-cf6d-42d3-b67e-c83a72e25727 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:17.445574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7582f45-cf6d-42d3-b67e-c83a72e25727 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:17.446223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7582f45-cf6d-42d3-b67e-c83a72e25727 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:17.447649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7582f45-cf6d-42d3-b67e-c83a72e25727 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:17.474793 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 589/1173] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Sep 19 14:41:17 2022] POST /identity/v3/projects => generated 361 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:41:19.686834 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a59867-e7c8-4dbb-8118-d3217c49ccc7 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:19.687398 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a59867-e7c8-4dbb-8118-d3217c49ccc7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:19.687398 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a59867-e7c8-4dbb-8118-d3217c49ccc7 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:19.688738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 585/1174] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:41:19.693925 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf5885-8e86-4707-95ee-8f69b318367c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:19.694352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf5885-8e86-4707-95ee-8f69b318367c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:19.694468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf5885-8e86-4707-95ee-8f69b318367c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:19.712688 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-7eaf5885-8e86-4707-95ee-8f69b318367c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:41:19.713855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7eaf5885-8e86-4707-95ee-8f69b318367c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:41:19.771884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:19.781647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 590/1175] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:41:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:41:20.054984 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a59867-e7c8-4dbb-8118-d3217c49ccc7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:20.156129 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a59867-e7c8-4dbb-8118-d3217c49ccc7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:20.168537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:20.189859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1d7ca-d713-4e37-b381-773694fdf78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:20.190502 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1d7ca-d713-4e37-b381-773694fdf78b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:20.190779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1d7ca-d713-4e37-b381-773694fdf78b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:20.191047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1d7ca-d713-4e37-b381-773694fdf78b None admin] PATH_INFO: `/v3/domains/fe3e6215cdba41c6b85be13a81e263d9` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:20.192168 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1d7ca-d713-4e37-b381-773694fdf78b None admin] RBAC: Authorizing `identity:get_domain(domain_id=fe3e6215cdba41c6b85be13a81e263d9)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:20.193596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1d7ca-d713-4e37-b381-773694fdf78b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:20.195136 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 586/1176] 10.222.0.60 () {56 vars in 1183 bytes} [Mon Sep 19 14:41:20 2022] GET /identity/v3/domains/fe3e6215cdba41c6b85be13a81e263d9 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:20.201627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf5885-8e86-4707-95ee-8f69b318367c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:20.297596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf5885-8e86-4707-95ee-8f69b318367c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:20.331034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1869b5a-3212-4a85-b8d1-ed442c0b9dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:20.331564 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1869b5a-3212-4a85-b8d1-ed442c0b9dc1 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:20.331867 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1869b5a-3212-4a85-b8d1-ed442c0b9dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:20.332146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1869b5a-3212-4a85-b8d1-ed442c0b9dc1 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:20.332799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1869b5a-3212-4a85-b8d1-ed442c0b9dc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:20.334212 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1869b5a-3212-4a85-b8d1-ed442c0b9dc1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:20.356018 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 591/1177] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:41:20 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:41:22.534233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bf087-8abe-4279-b9bf-20ed57c0ff54 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:22.534797 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bf087-8abe-4279-b9bf-20ed57c0ff54 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:22.534797 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7bf087-8abe-4279-b9bf-20ed57c0ff54 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:22.536148 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 587/1178] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:41:22.541421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09183967-bfb2-49fb-a778-a69051145c77 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:22.541861 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09183967-bfb2-49fb-a778-a69051145c77 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:22.541979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09183967-bfb2-49fb-a778-a69051145c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:22.561282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-09183967-bfb2-49fb-a778-a69051145c77 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:41:22.562473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-09183967-bfb2-49fb-a778-a69051145c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:41:22.607509 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:22.629421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 592/1179] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:41:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:41:22.897845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bf087-8abe-4279-b9bf-20ed57c0ff54 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:22.996922 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7bf087-8abe-4279-b9bf-20ed57c0ff54 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:23.030430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501e76b-e6b4-4c2c-a86f-9f6cb1586716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:23.031048 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501e76b-e6b4-4c2c-a86f-9f6cb1586716 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:23.031333 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501e76b-e6b4-4c2c-a86f-9f6cb1586716 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:23.031607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501e76b-e6b4-4c2c-a86f-9f6cb1586716 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:23.041994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501e76b-e6b4-4c2c-a86f-9f6cb1586716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-1501e76b-e6b4-4c2c-a86f-9f6cb1586716 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:41:23.046755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:41:23.047981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:41:23.048929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:41:23.048929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:41:23.048929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:41:23.048929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:41:23.048929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:41:23.048929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:41:23.048929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:41:23.048929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:41:23.048929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:41:23.048929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:41:23.048929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:41:23.048929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:41:23.048929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 588/1180] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:41:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:41:23.054292 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09183967-bfb2-49fb-a778-a69051145c77 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:23.156503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09183967-bfb2-49fb-a778-a69051145c77 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:23.189414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d25dfc-f9e5-4a73-a86c-095f5aece85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:23.190023 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d25dfc-f9e5-4a73-a86c-095f5aece85d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:23.190404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d25dfc-f9e5-4a73-a86c-095f5aece85d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:23.190687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d25dfc-f9e5-4a73-a86c-095f5aece85d None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:23.191343 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d25dfc-f9e5-4a73-a86c-095f5aece85d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:41:23.191646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d25dfc-f9e5-4a73-a86c-095f5aece85d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:23.193034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d25dfc-f9e5-4a73-a86c-095f5aece85d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:23.201401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 593/1181] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:41:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:23.207462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501e76b-e6b4-4c2c-a86f-9f6cb1586716 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:23.248796 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:23.306007 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501e76b-e6b4-4c2c-a86f-9f6cb1586716 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:23.338647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e412f-cc07-45a0-99d8-ec7951284ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:23.339265 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e412f-cc07-45a0-99d8-ec7951284ac4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:23.339547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e412f-cc07-45a0-99d8-ec7951284ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:23.340484 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e412f-cc07-45a0-99d8-ec7951284ac4 None admin] PATH_INFO: `/v3/users/1de8fd9695d24473a74c129def5ca0b3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:23.355818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e412f-cc07-45a0-99d8-ec7951284ac4 None admin] RBAC: Authorizing `identity:get_user(user_id=1de8fd9695d24473a74c129def5ca0b3)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:23.357303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e412f-cc07-45a0-99d8-ec7951284ac4 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:23.358918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 589/1182] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:41:23 2022] GET /identity/v3/users/1de8fd9695d24473a74c129def5ca0b3 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:23.365180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d25dfc-f9e5-4a73-a86c-095f5aece85d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:23.408383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:23.464479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d25dfc-f9e5-4a73-a86c-095f5aece85d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:23.499485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457d0af-b358-41a1-a6e4-9d9324d3d0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:23.500182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457d0af-b358-41a1-a6e4-9d9324d3d0f1 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:23.500464 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457d0af-b358-41a1-a6e4-9d9324d3d0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:23.500735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457d0af-b358-41a1-a6e4-9d9324d3d0f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:23.565883 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457d0af-b358-41a1-a6e4-9d9324d3d0f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:23.567317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457d0af-b358-41a1-a6e4-9d9324d3d0f1 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:23.606717 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:23.663122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 594/1183] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:41:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:41:23.670372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e412f-cc07-45a0-99d8-ec7951284ac4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:23.761493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:23.765928 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e412f-cc07-45a0-99d8-ec7951284ac4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:23.799459 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02bd41-b1a3-483d-bf63-dbe714e768a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:23.800117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02bd41-b1a3-483d-bf63-dbe714e768a3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:23.800400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02bd41-b1a3-483d-bf63-dbe714e768a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:23.800682 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02bd41-b1a3-483d-bf63-dbe714e768a3 None admin] PATH_INFO: `/v3/projects/37d722369b164f0d9da907d034e29d07` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:23.802050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02bd41-b1a3-483d-bf63-dbe714e768a3 None admin] RBAC: Authorizing `identity:get_project(project_id=37d722369b164f0d9da907d034e29d07)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:23.803482 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac02bd41-b1a3-483d-bf63-dbe714e768a3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:23.805124 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 590/1184] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:41:23 2022] GET /identity/v3/projects/37d722369b164f0d9da907d034e29d07 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:41:23.811413 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457d0af-b358-41a1-a6e4-9d9324d3d0f1 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:23.909421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457d0af-b358-41a1-a6e4-9d9324d3d0f1 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:23.942361 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b576f1-d451-461c-890d-c3fce379101e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:23.942889 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b576f1-d451-461c-890d-c3fce379101e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:23.943167 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b576f1-d451-461c-890d-c3fce379101e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:23.943438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b576f1-d451-461c-890d-c3fce379101e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:23.944179 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b576f1-d451-461c-890d-c3fce379101e None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=37d722369b164f0d9da907d034e29d07, user.id=1de8fd9695d24473a74c129def5ca0b3) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:41:23.944484 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b576f1-d451-461c-890d-c3fce379101e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:23.945891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b576f1-d451-461c-890d-c3fce379101e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:23.951634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 595/1185] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:41:23 2022] GET /identity/v3/role_assignments?user.id=1de8fd9695d24473a74c129def5ca0b3&scope.project.id=37d722369b164f0d9da907d034e29d07&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:26.082975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca9997-9c11-4b74-bc77-9716d47c6aa7 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:26.083506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca9997-9c11-4b74-bc77-9716d47c6aa7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:26.083506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca9997-9c11-4b74-bc77-9716d47c6aa7 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:26.084885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 591/1186] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:41:26.090952 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc20d1-b05a-4bf3-83c2-95575eb41f35 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:26.091338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc20d1-b05a-4bf3-83c2-95575eb41f35 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:26.091485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc20d1-b05a-4bf3-83c2-95575eb41f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:26.103997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:26.109451 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-73fc20d1-b05a-4bf3-83c2-95575eb41f35 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:41:26.110657 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-73fc20d1-b05a-4bf3-83c2-95575eb41f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:41:26.175576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 596/1187] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:41:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:41:26.446778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca9997-9c11-4b74-bc77-9716d47c6aa7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:26.544767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca9997-9c11-4b74-bc77-9716d47c6aa7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:26.577857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafb1b2-fda0-4bea-8630-dca0067d47a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:26.578464 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafb1b2-fda0-4bea-8630-dca0067d47a4 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:26.578741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafb1b2-fda0-4bea-8630-dca0067d47a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:26.579011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafb1b2-fda0-4bea-8630-dca0067d47a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:26.588717 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafb1b2-fda0-4bea-8630-dca0067d47a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-0eafb1b2-fda0-4bea-8630-dca0067d47a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:41:26.593394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:41:26.595893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:41:26.597090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:41:26.597090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:41:26.597090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:41:26.597090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:41:26.597090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:41:26.597090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:41:26.597090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:41:26.597090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:41:26.597090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:41:26.597090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:41:26.597090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:41:26.597090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:41:26.597090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 592/1188] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:41:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:41:26.600893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc20d1-b05a-4bf3-83c2-95575eb41f35 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:26.707957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc20d1-b05a-4bf3-83c2-95575eb41f35 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:26.740464 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a687b-91c5-4860-8e26-0a71e376a89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:26.741058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a687b-91c5-4860-8e26-0a71e376a89e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:26.741337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a687b-91c5-4860-8e26-0a71e376a89e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:26.741609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a687b-91c5-4860-8e26-0a71e376a89e None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:26.742256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a687b-91c5-4860-8e26-0a71e376a89e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:41:26.742556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a687b-91c5-4860-8e26-0a71e376a89e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:26.743936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a687b-91c5-4860-8e26-0a71e376a89e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:26.746617 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:26.753300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 597/1189] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:41:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:26.759818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafb1b2-fda0-4bea-8630-dca0067d47a4 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:26.786241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:26.859019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafb1b2-fda0-4bea-8630-dca0067d47a4 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:26.893665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ed038-cf19-433b-89af-d4cbc643a935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:26.894288 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ed038-cf19-433b-89af-d4cbc643a935 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:26.894562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ed038-cf19-433b-89af-d4cbc643a935 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:26.894944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ed038-cf19-433b-89af-d4cbc643a935 None admin] PATH_INFO: `/v3/users/1de8fd9695d24473a74c129def5ca0b3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:26.896079 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ed038-cf19-433b-89af-d4cbc643a935 None admin] RBAC: Authorizing `identity:get_user(user_id=1de8fd9695d24473a74c129def5ca0b3)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:26.897501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ed038-cf19-433b-89af-d4cbc643a935 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:26.899060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 593/1190] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:41:26 2022] GET /identity/v3/users/1de8fd9695d24473a74c129def5ca0b3 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:26.904940 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a687b-91c5-4860-8e26-0a71e376a89e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:27.003070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a687b-91c5-4860-8e26-0a71e376a89e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:27.035217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd122c-4621-4991-b13a-5840b4f474f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:27.035853 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd122c-4621-4991-b13a-5840b4f474f8 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:27.036141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd122c-4621-4991-b13a-5840b4f474f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:27.036418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd122c-4621-4991-b13a-5840b4f474f8 None admin] PATH_INFO: `/v3/projects/37d722369b164f0d9da907d034e29d07` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:27.037520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd122c-4621-4991-b13a-5840b4f474f8 None admin] RBAC: Authorizing `identity:get_project(project_id=37d722369b164f0d9da907d034e29d07)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:27.038943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd122c-4621-4991-b13a-5840b4f474f8 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:27.040589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 598/1191] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:41:26 2022] GET /identity/v3/projects/37d722369b164f0d9da907d034e29d07 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:27.046990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ed038-cf19-433b-89af-d4cbc643a935 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:27.138603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:27.145745 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ed038-cf19-433b-89af-d4cbc643a935 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:27.179495 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeaecfd-f0bd-4b8d-9ff0-7790c52b4068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:27.180230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaecfd-f0bd-4b8d-9ff0-7790c52b4068 None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:27.180513 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaecfd-f0bd-4b8d-9ff0-7790c52b4068 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:27.180787 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeaecfd-f0bd-4b8d-9ff0-7790c52b4068 None admin] PATH_INFO: `/v3/projects/37d722369b164f0d9da907d034e29d07/users/1de8fd9695d24473a74c129def5ca0b3/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:27.182701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeaecfd-f0bd-4b8d-9ff0-7790c52b4068 None admin] RBAC: Authorizing `identity:create_grant(project_id=37d722369b164f0d9da907d034e29d07, user_id=1de8fd9695d24473a74c129def5ca0b3, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:27.184229 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeaecfd-f0bd-4b8d-9ff0-7790c52b4068 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:27.196451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 594/1192] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:41:27 2022] PUT /identity/v3/projects/37d722369b164f0d9da907d034e29d07/users/1de8fd9695d24473a74c129def5ca0b3/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:41:29.365780 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb661a03-a1d9-4f79-a53c-1f18841271d6 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:29.366301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb661a03-a1d9-4f79-a53c-1f18841271d6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:29.366301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb661a03-a1d9-4f79-a53c-1f18841271d6 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:29.367591 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 599/1193] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:41:29.373552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa6b38-b6b4-4ca0-817f-8aecf8ba03b4 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:29.374005 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa6b38-b6b4-4ca0-817f-8aecf8ba03b4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:29.374120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa6b38-b6b4-4ca0-817f-8aecf8ba03b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:29.392481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-86aa6b38-b6b4-4ca0-817f-8aecf8ba03b4 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:41:29.393713 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-86aa6b38-b6b4-4ca0-817f-8aecf8ba03b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:41:29.458400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 595/1194] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:41:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:41:29.723782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb661a03-a1d9-4f79-a53c-1f18841271d6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:29.788722 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:29.823737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb661a03-a1d9-4f79-a53c-1f18841271d6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:29.858447 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b69f7-edca-4dd9-89a1-0842bfb40747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:29.859160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b69f7-edca-4dd9-89a1-0842bfb40747 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:29.859471 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b69f7-edca-4dd9-89a1-0842bfb40747 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:29.859925 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b69f7-edca-4dd9-89a1-0842bfb40747 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:29.872285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b69f7-edca-4dd9-89a1-0842bfb40747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-323b69f7-edca-4dd9-89a1-0842bfb40747 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:41:29.877022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:41:29.878168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:41:29.879308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:41:29.879308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:41:29.879308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:41:29.879308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:41:29.879308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:41:29.879308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:41:29.879308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:41:29.879308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:41:29.879308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:41:29.879308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:41:29.879308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:41:29.879308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:41:29.879308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 600/1195] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:41:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:41:29.884310 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa6b38-b6b4-4ca0-817f-8aecf8ba03b4 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:29.965395 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:29.982177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aa6b38-b6b4-4ca0-817f-8aecf8ba03b4 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:30.015130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12564924-d25f-4f02-8ea0-fd5760c64a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:30.015732 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12564924-d25f-4f02-8ea0-fd5760c64a7d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:30.016044 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12564924-d25f-4f02-8ea0-fd5760c64a7d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:30.016318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12564924-d25f-4f02-8ea0-fd5760c64a7d None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:30.016967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12564924-d25f-4f02-8ea0-fd5760c64a7d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:41:30.017269 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12564924-d25f-4f02-8ea0-fd5760c64a7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:30.018638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12564924-d25f-4f02-8ea0-fd5760c64a7d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:30.026485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 596/1196] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:41:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:30.032633 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b69f7-edca-4dd9-89a1-0842bfb40747 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:30.131138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b69f7-edca-4dd9-89a1-0842bfb40747 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:30.155372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:30.164680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b780c-3958-4b5c-aea6-9388b7beed94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:30.165305 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b780c-3958-4b5c-aea6-9388b7beed94 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:30.165585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b780c-3958-4b5c-aea6-9388b7beed94 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:30.165855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b780c-3958-4b5c-aea6-9388b7beed94 None admin] PATH_INFO: `/v3/users/1de8fd9695d24473a74c129def5ca0b3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:30.167023 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b780c-3958-4b5c-aea6-9388b7beed94 None admin] RBAC: Authorizing `identity:get_user(user_id=1de8fd9695d24473a74c129def5ca0b3)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:30.169024 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b780c-3958-4b5c-aea6-9388b7beed94 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:30.170570 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 601/1197] 10.222.0.60 () {56 vars in 1177 bytes} [Mon Sep 19 14:41:30 2022] GET /identity/v3/users/1de8fd9695d24473a74c129def5ca0b3 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:30.176809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12564924-d25f-4f02-8ea0-fd5760c64a7d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:30.273217 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12564924-d25f-4f02-8ea0-fd5760c64a7d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:30.306621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badbc82-bf58-408b-84d3-6fa828ea1362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:30.307222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badbc82-bf58-408b-84d3-6fa828ea1362 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:30.307507 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badbc82-bf58-408b-84d3-6fa828ea1362 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:30.307782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badbc82-bf58-408b-84d3-6fa828ea1362 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:30.322588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:30.375211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4badbc82-bf58-408b-84d3-6fa828ea1362 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:30.376699 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4badbc82-bf58-408b-84d3-6fa828ea1362 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:30.473097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 597/1198] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:41:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:41:30.479226 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b780c-3958-4b5c-aea6-9388b7beed94 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:30.575697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b780c-3958-4b5c-aea6-9388b7beed94 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:30.607578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10603de-27d0-4c63-bfc5-d4d5c4e7b98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:30.608218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10603de-27d0-4c63-bfc5-d4d5c4e7b98e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:30.608505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10603de-27d0-4c63-bfc5-d4d5c4e7b98e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:30.608775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10603de-27d0-4c63-bfc5-d4d5c4e7b98e None admin] PATH_INFO: `/v3/projects/37d722369b164f0d9da907d034e29d07` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:30.609860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10603de-27d0-4c63-bfc5-d4d5c4e7b98e None admin] RBAC: Authorizing `identity:get_project(project_id=37d722369b164f0d9da907d034e29d07)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:30.611278 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10603de-27d0-4c63-bfc5-d4d5c4e7b98e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:30.612907 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 602/1199] 10.222.0.60 () {56 vars in 1186 bytes} [Mon Sep 19 14:41:30 2022] GET /identity/v3/projects/37d722369b164f0d9da907d034e29d07 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:30.619037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badbc82-bf58-408b-84d3-6fa828ea1362 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:41:30.663931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:30.717013 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badbc82-bf58-408b-84d3-6fa828ea1362 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:41:30.759797 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc391a-07c7-4a21-9460-8632d3e0aa8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:41:30.760431 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc391a-07c7-4a21-9460-8632d3e0aa8e None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:41:30.760714 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc391a-07c7-4a21-9460-8632d3e0aa8e None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:41:30.760990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc391a-07c7-4a21-9460-8632d3e0aa8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:41:30.761740 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc391a-07c7-4a21-9460-8632d3e0aa8e None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=37d722369b164f0d9da907d034e29d07, user.id=1de8fd9695d24473a74c129def5ca0b3) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:41:30.762052 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc391a-07c7-4a21-9460-8632d3e0aa8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:41:30.763694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc391a-07c7-4a21-9460-8632d3e0aa8e None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:41:30.769782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 598/1200] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:41:30 2022] GET /identity/v3/role_assignments?user.id=1de8fd9695d24473a74c129def5ca0b3&scope.project.id=37d722369b164f0d9da907d034e29d07&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:41:40.603758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=69 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:41:40.767095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=45 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:27.433692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd05f0-c3f0-44be-8e01-77b17f56bc5a None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:27.434366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd05f0-c3f0-44be-8e01-77b17f56bc5a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:27.434366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd05f0-c3f0-44be-8e01-77b17f56bc5a None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:27.435983 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 603/1201] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:42:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:42:27.442905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63defbbf-5674-4175-b7e8-ab2a9c92a1e4 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:27.443427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63defbbf-5674-4175-b7e8-ab2a9c92a1e4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:27.443531 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63defbbf-5674-4175-b7e8-ab2a9c92a1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:27.465093 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-63defbbf-5674-4175-b7e8-ab2a9c92a1e4 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:42:27.466684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-63defbbf-5674-4175-b7e8-ab2a9c92a1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:42:27.559871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 599/1202] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:42:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:42:27.865639 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd05f0-c3f0-44be-8e01-77b17f56bc5a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:28.008957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd05f0-c3f0-44be-8e01-77b17f56bc5a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:28.053419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3040d6-f2d6-4551-b65b-a2b28a789936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:28.054062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3040d6-f2d6-4551-b65b-a2b28a789936 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:28.054345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3040d6-f2d6-4551-b65b-a2b28a789936 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:28.054663 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3040d6-f2d6-4551-b65b-a2b28a789936 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:28.069124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3040d6-f2d6-4551-b65b-a2b28a789936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:28.070744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3040d6-f2d6-4551-b65b-a2b28a789936 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-3b3040d6-f2d6-4551-b65b-a2b28a789936 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:42:28.097468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:28.098693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:42:28.099692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:42:28.099692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:28.099692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:28.099692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 604/1203] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:28.106482 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63defbbf-5674-4175-b7e8-ab2a9c92a1e4 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:28.245566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63defbbf-5674-4175-b7e8-ab2a9c92a1e4 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:28.290741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:28.298578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c48af-160e-4d1b-b4b8-4e92ca835d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:28.299358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c48af-160e-4d1b-b4b8-4e92ca835d7c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:28.299711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c48af-160e-4d1b-b4b8-4e92ca835d7c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:28.300116 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c48af-160e-4d1b-b4b8-4e92ca835d7c None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:28.300969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c48af-160e-4d1b-b4b8-4e92ca835d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:28.301339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c48af-160e-4d1b-b4b8-4e92ca835d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:28.302986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c48af-160e-4d1b-b4b8-4e92ca835d7c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:28.322138 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 600/1204] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:28.325098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3040d6-f2d6-4551-b65b-a2b28a789936 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:28.391452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:28.448697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3040d6-f2d6-4551-b65b-a2b28a789936 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:28.492809 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5bbbe7-6e14-4ff8-8c54-60e025295cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:28.493619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bbbe7-6e14-4ff8-8c54-60e025295cb3 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:28.494003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bbbe7-6e14-4ff8-8c54-60e025295cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:28.494309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5bbbe7-6e14-4ff8-8c54-60e025295cb3 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:28.498800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5bbbe7-6e14-4ff8-8c54-60e025295cb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:28.500247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5bbbe7-6e14-4ff8-8c54-60e025295cb3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:28.525119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 605/1205] 10.222.0.60 () {60 vars in 1132 bytes} [Mon Sep 19 14:42:28 2022] POST /identity/v3/users => generated 251 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:42:31.121924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5eb81-4e8d-405b-87c2-472956105d10 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:31.122508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5eb81-4e8d-405b-87c2-472956105d10 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:31.122604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5eb81-4e8d-405b-87c2-472956105d10 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:31.124013 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 601/1206] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:42:31.130848 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e5302-4a11-42c6-a958-6ba68a8f961d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:31.131345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e5302-4a11-42c6-a958-6ba68a8f961d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:31.131443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e5302-4a11-42c6-a958-6ba68a8f961d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:31.154786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-271e5302-4a11-42c6-a958-6ba68a8f961d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:42:31.158500 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-271e5302-4a11-42c6-a958-6ba68a8f961d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:42:31.263282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 606/1207] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:42:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:42:31.686121 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5eb81-4e8d-405b-87c2-472956105d10 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:31.819282 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5eb81-4e8d-405b-87c2-472956105d10 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:31.859548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f748543-1f23-4ef3-84c3-c6a17e882153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:31.860243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f748543-1f23-4ef3-84c3-c6a17e882153 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:31.860526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f748543-1f23-4ef3-84c3-c6a17e882153 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:31.860809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f748543-1f23-4ef3-84c3-c6a17e882153 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:31.872515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f748543-1f23-4ef3-84c3-c6a17e882153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-0f748543-1f23-4ef3-84c3-c6a17e882153 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:42:31.877307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:31.878302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:31.879413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:31.879413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:31.879413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:31.879413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:31.879413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:42:31.879413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:42:31.879413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:42:31.879413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:42:31.879413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:42:31.879413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:42:31.879413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:42:31.879413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:42:31.879413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 602/1208] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:42:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:31.885128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e5302-4a11-42c6-a958-6ba68a8f961d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:31.942001 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:32.030987 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e5302-4a11-42c6-a958-6ba68a8f961d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:32.088863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcd5d5-6856-4f24-b1a7-2c71ce5a1c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:32.090240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcd5d5-6856-4f24-b1a7-2c71ce5a1c1d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:32.090240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcd5d5-6856-4f24-b1a7-2c71ce5a1c1d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:32.090490 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcd5d5-6856-4f24-b1a7-2c71ce5a1c1d None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:32.091308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcd5d5-6856-4f24-b1a7-2c71ce5a1c1d None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:32.091616 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcd5d5-6856-4f24-b1a7-2c71ce5a1c1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:32.093611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcd5d5-6856-4f24-b1a7-2c71ce5a1c1d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:32.110383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 607/1209] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:42:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:42:32.115992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f748543-1f23-4ef3-84c3-c6a17e882153 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:32.175609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:32.240199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f748543-1f23-4ef3-84c3-c6a17e882153 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:32.283320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e958c7d-00a1-4376-8a10-d94b4c3b79ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:32.284081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e958c7d-00a1-4376-8a10-d94b4c3b79ce None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:32.284336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e958c7d-00a1-4376-8a10-d94b4c3b79ce None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:32.284648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e958c7d-00a1-4376-8a10-d94b4c3b79ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:32.298266 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e958c7d-00a1-4376-8a10-d94b4c3b79ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:32.299725 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e958c7d-00a1-4376-8a10-d94b4c3b79ce None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-1e958c7d-00a1-4376-8a10-d94b4c3b79ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:42:32.315956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:32.317224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:42:32.318339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:42:32.318339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:32.318339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:42:32.318339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 603/1210] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:42:32.325105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcd5d5-6856-4f24-b1a7-2c71ce5a1c1d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:32.447359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:32.451361 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcd5d5-6856-4f24-b1a7-2c71ce5a1c1d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:32.515929 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe71e9-b49d-415c-9675-2129d5a76564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:32.516591 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe71e9-b49d-415c-9675-2129d5a76564 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:32.516871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe71e9-b49d-415c-9675-2129d5a76564 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:32.517145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe71e9-b49d-415c-9675-2129d5a76564 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:32.517799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe71e9-b49d-415c-9675-2129d5a76564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:32.518101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe71e9-b49d-415c-9675-2129d5a76564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:32.519481 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe71e9-b49d-415c-9675-2129d5a76564 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:32.533484 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 608/1211] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:32.542828 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e958c7d-00a1-4376-8a10-d94b4c3b79ce None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:32.658392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e958c7d-00a1-4376-8a10-d94b4c3b79ce None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:32.660279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:32.711760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e746f-068d-4f98-af90-f6c1d6ee28ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:32.712616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e746f-068d-4f98-af90-f6c1d6ee28ad None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:32.712945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e746f-068d-4f98-af90-f6c1d6ee28ad None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:32.713705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e746f-068d-4f98-af90-f6c1d6ee28ad None admin] PATH_INFO: `/v3/users/placement` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:32.725443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e746f-068d-4f98-af90-f6c1d6ee28ad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:32.727197 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e746f-068d-4f98-af90-f6c1d6ee28ad None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-668e746f-068d-4f98-af90-f6c1d6ee28ad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:42:32.744382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:32.745592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:32.746698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:32.746698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:32.746698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:42:32.746698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:42:32.746698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:42:32.746698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:42:32.746698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:42:32.746698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:42:32.746698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 14:42:32.746698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:42:32.746698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 604/1212] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:32.753210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe71e9-b49d-415c-9675-2129d5a76564 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:32.895227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe71e9-b49d-415c-9675-2129d5a76564 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:32.944539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407155c3-0f4e-44cc-ae2a-8323a63dcc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:32.945155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407155c3-0f4e-44cc-ae2a-8323a63dcc94 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:32.945446 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407155c3-0f4e-44cc-ae2a-8323a63dcc94 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:32.945742 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407155c3-0f4e-44cc-ae2a-8323a63dcc94 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:32.946434 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407155c3-0f4e-44cc-ae2a-8323a63dcc94 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:32.946738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407155c3-0f4e-44cc-ae2a-8323a63dcc94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:32.948199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407155c3-0f4e-44cc-ae2a-8323a63dcc94 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:32.970814 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 609/1213] 10.222.0.60 () {56 vars in 1143 bytes} [Mon Sep 19 14:42:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:32.976383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e746f-068d-4f98-af90-f6c1d6ee28ad None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:33.116295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e746f-068d-4f98-af90-f6c1d6ee28ad None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:33.157865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3756d-3548-4b4a-a23c-c72036074002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:33.158764 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3756d-3548-4b4a-a23c-c72036074002 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:33.159047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3756d-3548-4b4a-a23c-c72036074002 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:33.159321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3756d-3548-4b4a-a23c-c72036074002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:33.199616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:33.243537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3756d-3548-4b4a-a23c-c72036074002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:33.245356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3756d-3548-4b4a-a23c-c72036074002 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:33.389481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 605/1214] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:42:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:42:33.395618 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407155c3-0f4e-44cc-ae2a-8323a63dcc94 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:33.424030 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:33.532559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407155c3-0f4e-44cc-ae2a-8323a63dcc94 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:33.576595 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d2bf9-4d85-43c1-8ad6-66acf246a404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:33.577226 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d2bf9-4d85-43c1-8ad6-66acf246a404 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:33.577504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d2bf9-4d85-43c1-8ad6-66acf246a404 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:33.577772 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d2bf9-4d85-43c1-8ad6-66acf246a404 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:33.589988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d2bf9-4d85-43c1-8ad6-66acf246a404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:33.591215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d2bf9-4d85-43c1-8ad6-66acf246a404 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-2a5d2bf9-4d85-43c1-8ad6-66acf246a404 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:42:33.608486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:33.609964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:42:33.611014 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:42:33.611014 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:33.611014 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:33.611014 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 610/1215] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:33.616290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3756d-3548-4b4a-a23c-c72036074002 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:33.707188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:33.766715 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3756d-3548-4b4a-a23c-c72036074002 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:33.816326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64855c-e657-4eb3-bb4b-24282a5fc44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:33.817169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64855c-e657-4eb3-bb4b-24282a5fc44b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:33.817564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64855c-e657-4eb3-bb4b-24282a5fc44b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:33.817947 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64855c-e657-4eb3-bb4b-24282a5fc44b None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:33.818845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64855c-e657-4eb3-bb4b-24282a5fc44b None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:33.819288 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64855c-e657-4eb3-bb4b-24282a5fc44b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:33.821316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64855c-e657-4eb3-bb4b-24282a5fc44b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:33.837858 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 606/1216] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:33.844229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d2bf9-4d85-43c1-8ad6-66acf246a404 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:33.920602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:33.956455 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d2bf9-4d85-43c1-8ad6-66acf246a404 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:33.993735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d52e8-02ac-4abc-8e83-05d278bbcc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:33.994781 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d52e8-02ac-4abc-8e83-05d278bbcc3e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:33.995030 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d52e8-02ac-4abc-8e83-05d278bbcc3e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:33.995303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d52e8-02ac-4abc-8e83-05d278bbcc3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:34.003068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d52e8-02ac-4abc-8e83-05d278bbcc3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:34.004867 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d52e8-02ac-4abc-8e83-05d278bbcc3e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:34.019073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-ea2d52e8-02ac-4abc-8e83-05d278bbcc3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:42:34.019073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:34.019073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:34.019073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:34.019073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:34.019073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:34.019073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:34.019073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:34.019073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:34.019073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:34.019073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:34.019073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:34.019073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:42:34.019073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:42:34.019073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:34.020122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:34.021123 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:42:34.021123 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:42:34.021123 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:42:34.021123 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:42:34.021123 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:42:34.021123 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:42:34.021123 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:42:34.021123 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:34.021123 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 611/1217] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:42:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:42:34.028884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64855c-e657-4eb3-bb4b-24282a5fc44b None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:34.149486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64855c-e657-4eb3-bb4b-24282a5fc44b None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:34.178509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:34.192054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde397c5-347e-4ee3-b54f-d8790bd4db88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:34.192709 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde397c5-347e-4ee3-b54f-d8790bd4db88 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:34.193024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde397c5-347e-4ee3-b54f-d8790bd4db88 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:34.193302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde397c5-347e-4ee3-b54f-d8790bd4db88 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:34.193994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde397c5-347e-4ee3-b54f-d8790bd4db88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:34.194298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde397c5-347e-4ee3-b54f-d8790bd4db88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:34.195730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde397c5-347e-4ee3-b54f-d8790bd4db88 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:34.212173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 607/1218] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:42:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:34.218891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d52e8-02ac-4abc-8e83-05d278bbcc3e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:34.331507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d52e8-02ac-4abc-8e83-05d278bbcc3e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:34.364911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:34.372918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460240b4-0ca5-4f41-abe7-8a312cd83453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:34.373797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460240b4-0ca5-4f41-abe7-8a312cd83453 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:34.374190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460240b4-0ca5-4f41-abe7-8a312cd83453 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:34.374600 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460240b4-0ca5-4f41-abe7-8a312cd83453 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:34.375610 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460240b4-0ca5-4f41-abe7-8a312cd83453 None admin] RBAC: Adding query filter params (role.id=15fbc4d519f64f4cacecc3b20fe826ae, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=568189358b154b66a729fae2f0411d1b) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:34.377473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460240b4-0ca5-4f41-abe7-8a312cd83453 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:34.377753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460240b4-0ca5-4f41-abe7-8a312cd83453 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:34.387732 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 612/1219] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:42:34 2022] GET /identity/v3/role_assignments?user.id=568189358b154b66a729fae2f0411d1b&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=15fbc4d519f64f4cacecc3b20fe826ae => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:42:36.692746 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94278490-d934-4fdc-9614-b15088ff7799 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:36.693466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94278490-d934-4fdc-9614-b15088ff7799 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:36.693466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94278490-d934-4fdc-9614-b15088ff7799 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:36.695019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 608/1220] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:42:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:42:36.701933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46a6bf-7d4e-4ca3-8217-52485b5dd717 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:36.702423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46a6bf-7d4e-4ca3-8217-52485b5dd717 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:36.702586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46a6bf-7d4e-4ca3-8217-52485b5dd717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:36.727415 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-cb46a6bf-7d4e-4ca3-8217-52485b5dd717 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:42:36.729216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cb46a6bf-7d4e-4ca3-8217-52485b5dd717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:42:36.810227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 613/1221] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:42:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:42:37.113265 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94278490-d934-4fdc-9614-b15088ff7799 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:37.241638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94278490-d934-4fdc-9614-b15088ff7799 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:37.285907 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dff74-2b3b-468e-9a37-50571eafa4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:37.286545 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dff74-2b3b-468e-9a37-50571eafa4b3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:37.286836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dff74-2b3b-468e-9a37-50571eafa4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:37.287180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971dff74-2b3b-468e-9a37-50571eafa4b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:37.300960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971dff74-2b3b-468e-9a37-50571eafa4b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-971dff74-2b3b-468e-9a37-50571eafa4b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:42:37.308099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:37.309261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:37.310156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:37.310156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:37.310156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:37.310156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:37.310156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:42:37.310156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:42:37.310156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:42:37.310156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:42:37.310156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:42:37.310156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:42:37.310156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:42:37.310156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:42:37.310156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 609/1222] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:42:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:37.316080 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46a6bf-7d4e-4ca3-8217-52485b5dd717 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:37.422421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46a6bf-7d4e-4ca3-8217-52485b5dd717 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:37.448239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:37.460685 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f06ee-e176-4c3a-85a2-929f113d9861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:37.461332 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f06ee-e176-4c3a-85a2-929f113d9861 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:37.461617 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f06ee-e176-4c3a-85a2-929f113d9861 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:37.461896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f06ee-e176-4c3a-85a2-929f113d9861 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:37.462565 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f06ee-e176-4c3a-85a2-929f113d9861 None admin] RBAC: Adding query filter params (name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:37.462866 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f06ee-e176-4c3a-85a2-929f113d9861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:37.464565 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f06ee-e176-4c3a-85a2-929f113d9861 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:37.475336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 614/1223] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:42:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:37.482257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dff74-2b3b-468e-9a37-50571eafa4b3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:37.535226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:37.595555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971dff74-2b3b-468e-9a37-50571eafa4b3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:37.636358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e744d07-56dd-4e64-b15e-933ad08ceb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:37.637063 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e744d07-56dd-4e64-b15e-933ad08ceb7f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:37.637384 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e744d07-56dd-4e64-b15e-933ad08ceb7f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:37.637708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e744d07-56dd-4e64-b15e-933ad08ceb7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:37.652107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e744d07-56dd-4e64-b15e-933ad08ceb7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:37.654099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e744d07-56dd-4e64-b15e-933ad08ceb7f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-1e744d07-56dd-4e64-b15e-933ad08ceb7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:42:37.671916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:37.673192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:42:37.674154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:42:37.674154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:37.674154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:42:37.674154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 610/1224] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:37.680845 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f06ee-e176-4c3a-85a2-929f113d9861 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:37.796511 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f06ee-e176-4c3a-85a2-929f113d9861 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:37.839426 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f9532-7624-4198-8158-767815f5e158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:37.840276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f9532-7624-4198-8158-767815f5e158 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:37.840678 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f9532-7624-4198-8158-767815f5e158 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:37.841107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f9532-7624-4198-8158-767815f5e158 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:37.841961 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f9532-7624-4198-8158-767815f5e158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:37.842386 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f9532-7624-4198-8158-767815f5e158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:37.844155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709f9532-7624-4198-8158-767815f5e158 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:37.860476 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 615/1225] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:42:37.866363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e744d07-56dd-4e64-b15e-933ad08ceb7f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:37.978182 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:37.985535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e744d07-56dd-4e64-b15e-933ad08ceb7f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:38.020727 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb330d7-df2e-4408-a2ca-1316a44c2e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:38.021353 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb330d7-df2e-4408-a2ca-1316a44c2e96 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:38.021698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb330d7-df2e-4408-a2ca-1316a44c2e96 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:38.021939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb330d7-df2e-4408-a2ca-1316a44c2e96 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:38.030179 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb330d7-df2e-4408-a2ca-1316a44c2e96 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:38.031642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb330d7-df2e-4408-a2ca-1316a44c2e96 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-9eb330d7-df2e-4408-a2ca-1316a44c2e96 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:42:38.042799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:38.044035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:38.045202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:38.045202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:38.045202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:42:38.045202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:42:38.045202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:42:38.045202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:42:38.045202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:42:38.045202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:42:38.045202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 14:42:38.045202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:42:38.045202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 611/1226] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:38.049710 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f9532-7624-4198-8158-767815f5e158 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:38.102374 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:38.160282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709f9532-7624-4198-8158-767815f5e158 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:38.199091 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ddc63-2dde-40a9-bffd-c28c9f791cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:38.199800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ddc63-2dde-40a9-bffd-c28c9f791cf4 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:38.200165 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ddc63-2dde-40a9-bffd-c28c9f791cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:38.200445 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ddc63-2dde-40a9-bffd-c28c9f791cf4 None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:38.201228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ddc63-2dde-40a9-bffd-c28c9f791cf4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:38.201556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ddc63-2dde-40a9-bffd-c28c9f791cf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:38.203122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ddc63-2dde-40a9-bffd-c28c9f791cf4 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:38.227980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 616/1227] 10.222.0.60 () {56 vars in 1143 bytes} [Mon Sep 19 14:42:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:38.229241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb330d7-df2e-4408-a2ca-1316a44c2e96 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:38.344541 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb330d7-df2e-4408-a2ca-1316a44c2e96 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:38.383485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee151be-05e1-4b7c-8b73-19bd282f3f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:38.384154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee151be-05e1-4b7c-8b73-19bd282f3f66 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:38.384449 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee151be-05e1-4b7c-8b73-19bd282f3f66 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:38.384723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee151be-05e1-4b7c-8b73-19bd282f3f66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:38.396382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee151be-05e1-4b7c-8b73-19bd282f3f66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:38.398123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee151be-05e1-4b7c-8b73-19bd282f3f66 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-fee151be-05e1-4b7c-8b73-19bd282f3f66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:42:38.414501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:38.415986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:42:38.417295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:42:38.417295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:38.417295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:42:38.417575 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 612/1228] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:38.423495 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ddc63-2dde-40a9-bffd-c28c9f791cf4 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:38.527567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ddc63-2dde-40a9-bffd-c28c9f791cf4 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:38.529913 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:38.569995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45e9e7-2b04-435e-a21a-682198dc703f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:38.570682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45e9e7-2b04-435e-a21a-682198dc703f None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:38.570993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45e9e7-2b04-435e-a21a-682198dc703f None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:38.571267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45e9e7-2b04-435e-a21a-682198dc703f None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:38.571935 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45e9e7-2b04-435e-a21a-682198dc703f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:38.572288 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45e9e7-2b04-435e-a21a-682198dc703f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:38.573840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45e9e7-2b04-435e-a21a-682198dc703f None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:38.592038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 617/1229] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:42:38.597954 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee151be-05e1-4b7c-8b73-19bd282f3f66 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:38.608139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:38.717163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee151be-05e1-4b7c-8b73-19bd282f3f66 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:38.754880 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14df48f-47e6-4b84-a110-629fe1db8a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:38.755521 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14df48f-47e6-4b84-a110-629fe1db8a65 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:38.755855 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14df48f-47e6-4b84-a110-629fe1db8a65 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:38.756165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14df48f-47e6-4b84-a110-629fe1db8a65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:38.764112 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14df48f-47e6-4b84-a110-629fe1db8a65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:38.765735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14df48f-47e6-4b84-a110-629fe1db8a65 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:38.778867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-f14df48f-47e6-4b84-a110-629fe1db8a65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:42:38.778867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:38.778867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:38.778867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:38.778867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:38.778867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:38.778867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:38.778867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:38.778867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:38.778867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:38.778867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:38.778867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:38.778867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:42:38.778867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:42:38.778867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:38.779990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:38.780977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:42:38.780977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:42:38.780977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:42:38.780977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:42:38.780977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:42:38.780977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:42:38.780977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:42:38.780977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:42:38.780977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 613/1230] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:42:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:38.786059 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45e9e7-2b04-435e-a21a-682198dc703f None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:38.900185 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45e9e7-2b04-435e-a21a-682198dc703f None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:38.940819 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619def89-e772-49f9-8a70-310ee61a62d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:38.941443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619def89-e772-49f9-8a70-310ee61a62d7 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:38.941724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619def89-e772-49f9-8a70-310ee61a62d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:38.941998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619def89-e772-49f9-8a70-310ee61a62d7 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:38.942709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619def89-e772-49f9-8a70-310ee61a62d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:38.943052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619def89-e772-49f9-8a70-310ee61a62d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:38.944605 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619def89-e772-49f9-8a70-310ee61a62d7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:38.958085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 618/1231] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:42:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:38.964161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14df48f-47e6-4b84-a110-629fe1db8a65 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:39.033273 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:39.069157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14df48f-47e6-4b84-a110-629fe1db8a65 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:39.106401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077da178-f564-4e8d-9e23-bbc7a44d8ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:39.107111 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077da178-f564-4e8d-9e23-bbc7a44d8ed1 None admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:39.107389 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077da178-f564-4e8d-9e23-bbc7a44d8ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:39.107701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077da178-f564-4e8d-9e23-bbc7a44d8ed1 None admin] PATH_INFO: `/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/568189358b154b66a729fae2f0411d1b/roles/15fbc4d519f64f4cacecc3b20fe826ae` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:39.124177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077da178-f564-4e8d-9e23-bbc7a44d8ed1 None admin] RBAC: Authorizing `identity:create_grant(project_id=7dee98edf19d478ca4f746ff85579fba, user_id=568189358b154b66a729fae2f0411d1b, role_id=15fbc4d519f64f4cacecc3b20fe826ae)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:39.125721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077da178-f564-4e8d-9e23-bbc7a44d8ed1 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:39.139392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 614/1232] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:42:38 2022] PUT /identity/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/568189358b154b66a729fae2f0411d1b/roles/15fbc4d519f64f4cacecc3b20fe826ae => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:42:41.335932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab1b48-c608-4362-8b85-6a87bad39688 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:41.336575 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab1b48-c608-4362-8b85-6a87bad39688 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:41.336575 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab1b48-c608-4362-8b85-6a87bad39688 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:41.337973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 619/1233] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:42:41.344498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45dabd5-ed88-4cb4-8ea1-a8ed58944700 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:41.345024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45dabd5-ed88-4cb4-8ea1-a8ed58944700 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:41.345024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45dabd5-ed88-4cb4-8ea1-a8ed58944700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:41.364031 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d45dabd5-ed88-4cb4-8ea1-a8ed58944700 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:42:41.365317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d45dabd5-ed88-4cb4-8ea1-a8ed58944700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:42:41.440369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 615/1234] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:42:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:42:41.768136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab1b48-c608-4362-8b85-6a87bad39688 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:41.795188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:41.877145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab1b48-c608-4362-8b85-6a87bad39688 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:41.915255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8661405-180f-4f3b-ad59-920300aa2db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:41.915968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8661405-180f-4f3b-ad59-920300aa2db3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:41.916313 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8661405-180f-4f3b-ad59-920300aa2db3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:41.916653 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8661405-180f-4f3b-ad59-920300aa2db3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:41.932409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8661405-180f-4f3b-ad59-920300aa2db3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-a8661405-180f-4f3b-ad59-920300aa2db3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:42:41.938105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:41.939052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:41.939971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:41.939971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:41.939971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:41.939971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:41.939971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:42:41.939971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:42:41.939971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:42:41.939971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:42:41.939971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:42:41.939971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:42:41.939971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 19 14:42:41.939971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:41.939971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 620/1235] 10.222.0.60 () {56 vars in 1102 bytes} [Mon Sep 19 14:42:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:41.946069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45dabd5-ed88-4cb4-8ea1-a8ed58944700 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:41.994641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:42.052835 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45dabd5-ed88-4cb4-8ea1-a8ed58944700 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:42.096497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272ef49-ed2e-4937-a2c8-1d5fbbdefab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:42.097172 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272ef49-ed2e-4937-a2c8-1d5fbbdefab9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:42.097510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272ef49-ed2e-4937-a2c8-1d5fbbdefab9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:42.097898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272ef49-ed2e-4937-a2c8-1d5fbbdefab9 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:42.098575 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b272ef49-ed2e-4937-a2c8-1d5fbbdefab9 None admin] RBAC: Adding query filter params (name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:42.098883 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b272ef49-ed2e-4937-a2c8-1d5fbbdefab9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:42.100298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b272ef49-ed2e-4937-a2c8-1d5fbbdefab9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:42.110392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 616/1236] 10.222.0.60 () {56 vars in 1103 bytes} [Mon Sep 19 14:42:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:42.116359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8661405-180f-4f3b-ad59-920300aa2db3 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:42.212572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:42.229158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8661405-180f-4f3b-ad59-920300aa2db3 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:42.271941 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74623acf-296a-4cc8-9dd8-69aa3f68787e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:42.272578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74623acf-296a-4cc8-9dd8-69aa3f68787e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:42.272911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74623acf-296a-4cc8-9dd8-69aa3f68787e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:42.273212 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74623acf-296a-4cc8-9dd8-69aa3f68787e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:42.285216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74623acf-296a-4cc8-9dd8-69aa3f68787e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:42.286822 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74623acf-296a-4cc8-9dd8-69aa3f68787e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-74623acf-296a-4cc8-9dd8-69aa3f68787e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:42:42.303700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:42.304849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:42:42.305774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:42:42.305774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:42.305774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:42.305774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 621/1237] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:42.311908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272ef49-ed2e-4937-a2c8-1d5fbbdefab9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:42.424670 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272ef49-ed2e-4937-a2c8-1d5fbbdefab9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:42.435420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:42.462557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1b443-5673-4014-8b7b-43f6f3c8c500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:42.463298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1b443-5673-4014-8b7b-43f6f3c8c500 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:42.463712 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1b443-5673-4014-8b7b-43f6f3c8c500 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:42.464028 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1b443-5673-4014-8b7b-43f6f3c8c500 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:42.464726 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1b443-5673-4014-8b7b-43f6f3c8c500 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:42.465190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1b443-5673-4014-8b7b-43f6f3c8c500 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:42.467424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1b443-5673-4014-8b7b-43f6f3c8c500 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:42.486295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 617/1238] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:42.494917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74623acf-296a-4cc8-9dd8-69aa3f68787e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:42.607502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74623acf-296a-4cc8-9dd8-69aa3f68787e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:42.643205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3e3e4-eefa-4980-877f-b8f48df8a176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:42.643889 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3e3e4-eefa-4980-877f-b8f48df8a176 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:42.644244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3e3e4-eefa-4980-877f-b8f48df8a176 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:42.644464 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3e3e4-eefa-4980-877f-b8f48df8a176 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:42.647747 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:42.653229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3e3e4-eefa-4980-877f-b8f48df8a176 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:42.654872 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3e3e4-eefa-4980-877f-b8f48df8a176 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-30f3e3e4-eefa-4980-877f-b8f48df8a176 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:42:42.670002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:42.671621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:42.672839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:42.672839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:42.672839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:42:42.672839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:42:42.672839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:42:42.672839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:42:42.672839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:42:42.672839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:42:42.672839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 14:42:42.672839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:42.672839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 622/1239] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 19 14:42:42.678434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1b443-5673-4014-8b7b-43f6f3c8c500 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:42.810892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1b443-5673-4014-8b7b-43f6f3c8c500 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:42.854029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6459f0-a963-496c-b668-9ab1757b06e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:42.854736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6459f0-a963-496c-b668-9ab1757b06e9 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:42.855029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6459f0-a963-496c-b668-9ab1757b06e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:42.855391 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6459f0-a963-496c-b668-9ab1757b06e9 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:42.856050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6459f0-a963-496c-b668-9ab1757b06e9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:42.856393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6459f0-a963-496c-b668-9ab1757b06e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:42.858020 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6459f0-a963-496c-b668-9ab1757b06e9 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:42.891465 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 618/1240] 10.222.0.60 () {56 vars in 1143 bytes} [Mon Sep 19 14:42:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:42.898547 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3e3e4-eefa-4980-877f-b8f48df8a176 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:43.017142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3e3e4-eefa-4980-877f-b8f48df8a176 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:43.055137 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41430b96-8cfd-4096-8263-b68330b18c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:43.055743 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41430b96-8cfd-4096-8263-b68330b18c52 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:43.056055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41430b96-8cfd-4096-8263-b68330b18c52 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:43.056317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41430b96-8cfd-4096-8263-b68330b18c52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:43.136356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:43.147552 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41430b96-8cfd-4096-8263-b68330b18c52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:43.149157 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41430b96-8cfd-4096-8263-b68330b18c52 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:43.264892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 623/1241] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:42:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:42:43.271716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6459f0-a963-496c-b668-9ab1757b06e9 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:43.303491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:43.392262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6459f0-a963-496c-b668-9ab1757b06e9 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:43.437375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982d13b-300c-4f12-92f9-18b1e31e821a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:43.438378 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982d13b-300c-4f12-92f9-18b1e31e821a None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:43.438803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982d13b-300c-4f12-92f9-18b1e31e821a None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:43.439179 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982d13b-300c-4f12-92f9-18b1e31e821a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:43.451443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982d13b-300c-4f12-92f9-18b1e31e821a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:43.452990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1982d13b-300c-4f12-92f9-18b1e31e821a None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-1982d13b-300c-4f12-92f9-18b1e31e821a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:42:43.468373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:43.469512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:42:43.470642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:42:43.470642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:43.470642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:42:43.470642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 619/1242] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:43.475845 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41430b96-8cfd-4096-8263-b68330b18c52 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:43.578924 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:43.585444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41430b96-8cfd-4096-8263-b68330b18c52 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:43.621621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce33eb-a4ba-4ff3-b323-55b28ed0eb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:43.622242 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce33eb-a4ba-4ff3-b323-55b28ed0eb72 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:43.622508 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce33eb-a4ba-4ff3-b323-55b28ed0eb72 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:43.622823 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce33eb-a4ba-4ff3-b323-55b28ed0eb72 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:43.623439 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce33eb-a4ba-4ff3-b323-55b28ed0eb72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:43.623773 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce33eb-a4ba-4ff3-b323-55b28ed0eb72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:43.625197 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fce33eb-a4ba-4ff3-b323-55b28ed0eb72 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:43.640683 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 624/1243] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:42:43.647267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982d13b-300c-4f12-92f9-18b1e31e821a None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:43.745026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:43.769051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1982d13b-300c-4f12-92f9-18b1e31e821a None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:43.805294 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a809b6d-07d6-4ba0-bf07-d9f86351fe2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:43.805972 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a809b6d-07d6-4ba0-bf07-d9f86351fe2f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:43.806252 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a809b6d-07d6-4ba0-bf07-d9f86351fe2f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:43.807059 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a809b6d-07d6-4ba0-bf07-d9f86351fe2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:43.814317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a809b6d-07d6-4ba0-bf07-d9f86351fe2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:43.815799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a809b6d-07d6-4ba0-bf07-d9f86351fe2f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:43.828125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-3a809b6d-07d6-4ba0-bf07-d9f86351fe2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:42:43.828125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:43.828125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:43.828125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:43.828125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:43.828125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:43.828125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:43.828125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:43.828125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:43.828125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:43.828125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:43.828125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:43.828125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:42:43.828125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:42:43.828125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:43.829263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:43.830193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:42:43.830193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:42:43.830193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:42:43.830193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:42:43.830193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:42:43.830193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:42:43.830193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:42:43.830193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:42:43.830193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 620/1244] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:42:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:43.835688 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce33eb-a4ba-4ff3-b323-55b28ed0eb72 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:43.971090 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fce33eb-a4ba-4ff3-b323-55b28ed0eb72 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:44.015035 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88218451-7026-4cf9-bd35-0f4751ef6e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:44.015704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88218451-7026-4cf9-bd35-0f4751ef6e48 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:44.016051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88218451-7026-4cf9-bd35-0f4751ef6e48 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:44.016312 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88218451-7026-4cf9-bd35-0f4751ef6e48 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:44.017004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88218451-7026-4cf9-bd35-0f4751ef6e48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:44.017313 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88218451-7026-4cf9-bd35-0f4751ef6e48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:44.019116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88218451-7026-4cf9-bd35-0f4751ef6e48 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:44.033612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 625/1245] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:42:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:44.041987 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a809b6d-07d6-4ba0-bf07-d9f86351fe2f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:44.164191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a809b6d-07d6-4ba0-bf07-d9f86351fe2f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:44.204147 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace99edc-94ae-43a7-a5de-9edecfb88236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:44.204754 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace99edc-94ae-43a7-a5de-9edecfb88236 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:44.205035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace99edc-94ae-43a7-a5de-9edecfb88236 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:44.205325 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace99edc-94ae-43a7-a5de-9edecfb88236 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:44.206019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace99edc-94ae-43a7-a5de-9edecfb88236 None admin] RBAC: Adding query filter params (role.id=15fbc4d519f64f4cacecc3b20fe826ae, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=568189358b154b66a729fae2f0411d1b) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:44.208288 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace99edc-94ae-43a7-a5de-9edecfb88236 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:44.208288 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace99edc-94ae-43a7-a5de-9edecfb88236 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:44.214513 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 621/1246] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:42:44 2022] GET /identity/v3/role_assignments?user.id=568189358b154b66a729fae2f0411d1b&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=15fbc4d519f64f4cacecc3b20fe826ae => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:42:46.749441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9d805-50f6-4c0d-8295-a1a06b373278 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:46.750008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9d805-50f6-4c0d-8295-a1a06b373278 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:46.750008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9d805-50f6-4c0d-8295-a1a06b373278 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:46.751510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 626/1247] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:42:46.757343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed222b06-bdfc-4b2b-a11f-72247137a3b7 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:46.757875 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed222b06-bdfc-4b2b-a11f-72247137a3b7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:46.757875 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed222b06-bdfc-4b2b-a11f-72247137a3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:46.764311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:46.777258 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ed222b06-bdfc-4b2b-a11f-72247137a3b7 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:42:46.778459 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ed222b06-bdfc-4b2b-a11f-72247137a3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:42:46.851460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 622/1248] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:42:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:42:47.130653 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9d805-50f6-4c0d-8295-a1a06b373278 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:47.149873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:47.241487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9d805-50f6-4c0d-8295-a1a06b373278 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:47.278621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e8e9b-d059-432a-8632-786acc265f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:47.279318 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e8e9b-d059-432a-8632-786acc265f78 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:47.279550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e8e9b-d059-432a-8632-786acc265f78 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:47.279865 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e8e9b-d059-432a-8632-786acc265f78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:47.292127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e8e9b-d059-432a-8632-786acc265f78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-ca6e8e9b-d059-432a-8632-786acc265f78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:42:47.296973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:47.298152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:47.299051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:47.299051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:47.299051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:47.299051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:47.299051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:42:47.299051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:42:47.299051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:42:47.299051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:42:47.299051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:42:47.299051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:42:47.299051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:42:47.299051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:47.299051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 627/1249] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:42:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:47.304560 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed222b06-bdfc-4b2b-a11f-72247137a3b7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:47.409905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed222b06-bdfc-4b2b-a11f-72247137a3b7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:47.442137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:47.448161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6c2be-7471-4a91-9a4d-b79b624a2391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:47.448790 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6c2be-7471-4a91-9a4d-b79b624a2391 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:47.449069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6c2be-7471-4a91-9a4d-b79b624a2391 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:47.449347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6c2be-7471-4a91-9a4d-b79b624a2391 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:47.450013 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6c2be-7471-4a91-9a4d-b79b624a2391 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:47.450313 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6c2be-7471-4a91-9a4d-b79b624a2391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:47.451708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6c2be-7471-4a91-9a4d-b79b624a2391 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:47.465649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 623/1250] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:42:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:47.466969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e8e9b-d059-432a-8632-786acc265f78 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:47.545302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:47.571258 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e8e9b-d059-432a-8632-786acc265f78 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:47.606723 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882f2c9-acdb-42b7-8f41-b59e95004c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:47.607355 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882f2c9-acdb-42b7-8f41-b59e95004c63 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:47.607634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882f2c9-acdb-42b7-8f41-b59e95004c63 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:47.607947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882f2c9-acdb-42b7-8f41-b59e95004c63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:47.618297 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882f2c9-acdb-42b7-8f41-b59e95004c63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:47.620345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882f2c9-acdb-42b7-8f41-b59e95004c63 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-a882f2c9-acdb-42b7-8f41-b59e95004c63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:42:47.643757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:47.645061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:42:47.646082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:42:47.646082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:47.646082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:47.646082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 628/1251] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:47.654461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6c2be-7471-4a91-9a4d-b79b624a2391 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:47.770479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6c2be-7471-4a91-9a4d-b79b624a2391 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:47.803655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6019b2-3872-4e91-83c3-96cb0e66f0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:47.804291 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6019b2-3872-4e91-83c3-96cb0e66f0a0 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:47.804574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6019b2-3872-4e91-83c3-96cb0e66f0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:47.804846 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6019b2-3872-4e91-83c3-96cb0e66f0a0 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:47.805476 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6019b2-3872-4e91-83c3-96cb0e66f0a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:47.805777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6019b2-3872-4e91-83c3-96cb0e66f0a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:47.807801 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6019b2-3872-4e91-83c3-96cb0e66f0a0 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:47.819177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 624/1252] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:47.825009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882f2c9-acdb-42b7-8f41-b59e95004c63 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:47.928124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882f2c9-acdb-42b7-8f41-b59e95004c63 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:47.956330 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:47.964386 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3034b-d32c-4a76-b001-c33270683668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:47.965017 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3034b-d32c-4a76-b001-c33270683668 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:47.965388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3034b-d32c-4a76-b001-c33270683668 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:47.965609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3034b-d32c-4a76-b001-c33270683668 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:47.973045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3034b-d32c-4a76-b001-c33270683668 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:47.974468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3034b-d32c-4a76-b001-c33270683668 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-a6a3034b-d32c-4a76-b001-c33270683668 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:42:47.985943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:47.987069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:47.988253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:47.988253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:47.988253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:42:47.988253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:42:47.988253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:42:47.988253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:42:47.988253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:42:47.988253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:42:47.988253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 14:42:47.988253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:47.988253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 629/1253] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:47.993000 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6019b2-3872-4e91-83c3-96cb0e66f0a0 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:48.047169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:48.092076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6019b2-3872-4e91-83c3-96cb0e66f0a0 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:48.126225 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdbb2f-083f-4eab-87f2-7cfec05d70a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:48.126877 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdbb2f-083f-4eab-87f2-7cfec05d70a3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:48.127154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdbb2f-083f-4eab-87f2-7cfec05d70a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:48.127429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdbb2f-083f-4eab-87f2-7cfec05d70a3 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:48.128178 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdbb2f-083f-4eab-87f2-7cfec05d70a3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:48.128484 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdbb2f-083f-4eab-87f2-7cfec05d70a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:48.129910 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdbb2f-083f-4eab-87f2-7cfec05d70a3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:48.146637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 625/1254] 10.222.0.60 () {56 vars in 1143 bytes} [Mon Sep 19 14:42:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:48.153028 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3034b-d32c-4a76-b001-c33270683668 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:48.253163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3034b-d32c-4a76-b001-c33270683668 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:48.286040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6a14e-032a-4d46-aad2-feb78bbac885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:48.286638 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6a14e-032a-4d46-aad2-feb78bbac885 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:48.286920 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6a14e-032a-4d46-aad2-feb78bbac885 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:48.287194 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6a14e-032a-4d46-aad2-feb78bbac885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:48.341019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:48.353897 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6a14e-032a-4d46-aad2-feb78bbac885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:48.355382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a6a14e-032a-4d46-aad2-feb78bbac885 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:48.450001 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 630/1255] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:42:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:42:48.456027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdbb2f-083f-4eab-87f2-7cfec05d70a3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:48.556169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdbb2f-083f-4eab-87f2-7cfec05d70a3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:48.568973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:48.590672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b3f73-c689-4247-a95c-4e882bea3c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:48.591363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b3f73-c689-4247-a95c-4e882bea3c4d None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:48.591587 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b3f73-c689-4247-a95c-4e882bea3c4d None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:48.591887 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b3f73-c689-4247-a95c-4e882bea3c4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:48.601785 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b3f73-c689-4247-a95c-4e882bea3c4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:48.603229 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b3f73-c689-4247-a95c-4e882bea3c4d None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-2a9b3f73-c689-4247-a95c-4e882bea3c4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:42:48.617420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:48.618621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:42:48.619724 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:42:48.619724 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:48.619724 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:42:48.619724 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 626/1256] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:48.624584 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6a14e-032a-4d46-aad2-feb78bbac885 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:48.712232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:48.724596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a6a14e-032a-4d46-aad2-feb78bbac885 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:48.757088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298aa728-e80b-477a-a95b-116858cf77cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:48.757697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298aa728-e80b-477a-a95b-116858cf77cd None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:48.757980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298aa728-e80b-477a-a95b-116858cf77cd None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:48.758251 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298aa728-e80b-477a-a95b-116858cf77cd None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:48.758886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298aa728-e80b-477a-a95b-116858cf77cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:48.759190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298aa728-e80b-477a-a95b-116858cf77cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:48.761119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298aa728-e80b-477a-a95b-116858cf77cd None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:48.772745 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 631/1257] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:48.778993 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b3f73-c689-4247-a95c-4e882bea3c4d None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:48.878361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b3f73-c689-4247-a95c-4e882bea3c4d None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:48.911814 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f375a59-7ef9-4112-90e8-1fca3c945800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:48.912461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f375a59-7ef9-4112-90e8-1fca3c945800 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:48.912742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f375a59-7ef9-4112-90e8-1fca3c945800 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:48.913019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f375a59-7ef9-4112-90e8-1fca3c945800 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:48.919478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f375a59-7ef9-4112-90e8-1fca3c945800 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:48.920970 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f375a59-7ef9-4112-90e8-1fca3c945800 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:48.932969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-3f375a59-7ef9-4112-90e8-1fca3c945800 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:42:48.932969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:48.932969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:48.932969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:48.932969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:48.932969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:48.932969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:48.932969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:48.932969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:48.932969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:48.932969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:48.932969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:48.932969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:42:48.932969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:42:48.932969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:48.934058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:48.935219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:42:48.935219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:42:48.935219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:42:48.935219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:42:48.935219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:42:48.935219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:42:48.935219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:42:48.935219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:42:48.935219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 627/1258] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:42:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:48.940233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298aa728-e80b-477a-a95b-116858cf77cd None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:49.039886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298aa728-e80b-477a-a95b-116858cf77cd None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:49.073118 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f367a-bcea-4586-ac66-0d46f404a187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:49.073717 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f367a-bcea-4586-ac66-0d46f404a187 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:49.074000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f367a-bcea-4586-ac66-0d46f404a187 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:49.074269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f367a-bcea-4586-ac66-0d46f404a187 None admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:49.074949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f367a-bcea-4586-ac66-0d46f404a187 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:49.075251 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f367a-bcea-4586-ac66-0d46f404a187 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:49.076701 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f367a-bcea-4586-ac66-0d46f404a187 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:49.088808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 632/1259] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:42:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:49.094884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f375a59-7ef9-4112-90e8-1fca3c945800 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:49.120422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:49.193246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f375a59-7ef9-4112-90e8-1fca3c945800 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:49.225968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2e61b-a551-4978-bda3-458eafae7cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:49.226565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2e61b-a551-4978-bda3-458eafae7cc1 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:49.226847 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2e61b-a551-4978-bda3-458eafae7cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:49.227120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2e61b-a551-4978-bda3-458eafae7cc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:49.227865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2e61b-a551-4978-bda3-458eafae7cc1 None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=568189358b154b66a729fae2f0411d1b) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:49.228167 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2e61b-a551-4978-bda3-458eafae7cc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:49.230146 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2e61b-a551-4978-bda3-458eafae7cc1 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:49.235863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 628/1260] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:42:49 2022] GET /identity/v3/role_assignments?user.id=568189358b154b66a729fae2f0411d1b&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:51.378435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51431d9-c887-4d08-844e-bffd1d06a2f9 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:51.378958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51431d9-c887-4d08-844e-bffd1d06a2f9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:51.378958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51431d9-c887-4d08-844e-bffd1d06a2f9 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:51.380275 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 633/1261] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:42:51.386061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542d412-cb78-4721-8514-e6a0e163b1b7 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:51.386489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542d412-cb78-4721-8514-e6a0e163b1b7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:51.386600 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542d412-cb78-4721-8514-e6a0e163b1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:51.404281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5542d412-cb78-4721-8514-e6a0e163b1b7 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:42:51.405449 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5542d412-cb78-4721-8514-e6a0e163b1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:42:51.469097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 629/1262] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:42:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:42:51.747259 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51431d9-c887-4d08-844e-bffd1d06a2f9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:51.758175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:51.848084 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51431d9-c887-4d08-844e-bffd1d06a2f9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:51.881692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c15b6c-3941-45f9-aeb9-6e8c30f0b81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:51.882340 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c15b6c-3941-45f9-aeb9-6e8c30f0b81d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:51.882620 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c15b6c-3941-45f9-aeb9-6e8c30f0b81d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:51.882894 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c15b6c-3941-45f9-aeb9-6e8c30f0b81d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:51.893531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c15b6c-3941-45f9-aeb9-6e8c30f0b81d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-c8c15b6c-3941-45f9-aeb9-6e8c30f0b81d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:42:51.898282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:51.899388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:51.901022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:51.901022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:51.901022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:51.901022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:51.901022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:42:51.901022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:42:51.901022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:42:51.901022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:42:51.901022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:42:51.901022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:42:51.901022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:42:51.901022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:51.901022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 634/1263] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:42:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:51.906118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542d412-cb78-4721-8514-e6a0e163b1b7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:51.924530 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:52.021628 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542d412-cb78-4721-8514-e6a0e163b1b7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:52.067060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ab21c-c4f8-4d35-afa2-9d34011cc372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:52.067663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ab21c-c4f8-4d35-afa2-9d34011cc372 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:52.067986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ab21c-c4f8-4d35-afa2-9d34011cc372 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:52.068259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ab21c-c4f8-4d35-afa2-9d34011cc372 None admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:52.068915 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ab21c-c4f8-4d35-afa2-9d34011cc372 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:52.069214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ab21c-c4f8-4d35-afa2-9d34011cc372 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:52.070589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898ab21c-c4f8-4d35-afa2-9d34011cc372 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:52.079241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 630/1264] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:42:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:52.084950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c15b6c-3941-45f9-aeb9-6e8c30f0b81d None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:52.152472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:52.185555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c15b6c-3941-45f9-aeb9-6e8c30f0b81d None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:52.218655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87462aad-e8b7-470e-b7b1-9c240da6b626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:52.219276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87462aad-e8b7-470e-b7b1-9c240da6b626 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:52.219560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87462aad-e8b7-470e-b7b1-9c240da6b626 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:52.219849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87462aad-e8b7-470e-b7b1-9c240da6b626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:52.230100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87462aad-e8b7-470e-b7b1-9c240da6b626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:52.231528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87462aad-e8b7-470e-b7b1-9c240da6b626 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-87462aad-e8b7-470e-b7b1-9c240da6b626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:42:52.247206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:52.248580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:42:52.249611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:42:52.249611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:52.249611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:52.249611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 635/1265] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:52.254624 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ab21c-c4f8-4d35-afa2-9d34011cc372 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:52.335717 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:52.358196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898ab21c-c4f8-4d35-afa2-9d34011cc372 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:52.391804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3c4cd-e529-4d0d-8833-56bb39829b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:52.392510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3c4cd-e529-4d0d-8833-56bb39829b21 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:52.392694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3c4cd-e529-4d0d-8833-56bb39829b21 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:52.392961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3c4cd-e529-4d0d-8833-56bb39829b21 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:52.393586 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3c4cd-e529-4d0d-8833-56bb39829b21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:52.393881 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3c4cd-e529-4d0d-8833-56bb39829b21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:52.395241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3c4cd-e529-4d0d-8833-56bb39829b21 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:52.406649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 631/1266] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:52.412334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87462aad-e8b7-470e-b7b1-9c240da6b626 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:52.508095 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87462aad-e8b7-470e-b7b1-9c240da6b626 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:52.527067 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:52.541567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43186d-576e-4708-b7c3-d884b5bf5647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:52.542188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43186d-576e-4708-b7c3-d884b5bf5647 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:52.542462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43186d-576e-4708-b7c3-d884b5bf5647 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:52.542729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43186d-576e-4708-b7c3-d884b5bf5647 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:52.549949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43186d-576e-4708-b7c3-d884b5bf5647 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:52.551363 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43186d-576e-4708-b7c3-d884b5bf5647 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-1b43186d-576e-4708-b7c3-d884b5bf5647 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:42:52.562950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:52.564178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:52.565362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:52.565362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:52.565362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:42:52.565362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:42:52.565362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:42:52.565362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:42:52.565362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:42:52.565362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:42:52.565362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 14:42:52.565362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:52.565362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 636/1267] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:52.569847 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3c4cd-e529-4d0d-8833-56bb39829b21 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:52.679122 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3c4cd-e529-4d0d-8833-56bb39829b21 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:52.713870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6c0ad-8fab-4ba4-8020-41a061d64658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:52.714474 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6c0ad-8fab-4ba4-8020-41a061d64658 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:52.714750 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6c0ad-8fab-4ba4-8020-41a061d64658 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:52.715022 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6c0ad-8fab-4ba4-8020-41a061d64658 None admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:52.715747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6c0ad-8fab-4ba4-8020-41a061d64658 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:52.716079 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6c0ad-8fab-4ba4-8020-41a061d64658 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:52.717512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6c0ad-8fab-4ba4-8020-41a061d64658 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:52.726476 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:52.735876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 632/1268] 10.222.0.60 () {56 vars in 1143 bytes} [Mon Sep 19 14:42:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:52.741948 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43186d-576e-4708-b7c3-d884b5bf5647 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:52.840608 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43186d-576e-4708-b7c3-d884b5bf5647 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:52.873887 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258da51-b2c9-4d68-be32-6210e3527986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:52.874516 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258da51-b2c9-4d68-be32-6210e3527986 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:52.874796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258da51-b2c9-4d68-be32-6210e3527986 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:52.875067 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258da51-b2c9-4d68-be32-6210e3527986 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:52.885673 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258da51-b2c9-4d68-be32-6210e3527986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:52.887117 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258da51-b2c9-4d68-be32-6210e3527986 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-5258da51-b2c9-4d68-be32-6210e3527986 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:42:52.901598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:52.902751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:42:52.904032 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:42:52.904032 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:52.904032 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:52.904032 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 637/1269] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:52.908595 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6c0ad-8fab-4ba4-8020-41a061d64658 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:53.006654 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6c0ad-8fab-4ba4-8020-41a061d64658 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:53.038803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69c69a-5af8-496b-a8b0-04281898eda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:53.039418 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69c69a-5af8-496b-a8b0-04281898eda7 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:53.039687 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69c69a-5af8-496b-a8b0-04281898eda7 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:53.039990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69c69a-5af8-496b-a8b0-04281898eda7 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:53.041175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69c69a-5af8-496b-a8b0-04281898eda7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:53.041479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69c69a-5af8-496b-a8b0-04281898eda7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:53.042840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69c69a-5af8-496b-a8b0-04281898eda7 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:53.053909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 633/1270] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:53.059791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258da51-b2c9-4d68-be32-6210e3527986 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:53.093354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:53.161497 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258da51-b2c9-4d68-be32-6210e3527986 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:53.199478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3ecea-0244-4a9e-98f9-be8a771f90ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:53.200128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3ecea-0244-4a9e-98f9-be8a771f90ff None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:53.200409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3ecea-0244-4a9e-98f9-be8a771f90ff None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:53.200680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd3ecea-0244-4a9e-98f9-be8a771f90ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:53.207942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3ecea-0244-4a9e-98f9-be8a771f90ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:53.209373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd3ecea-0244-4a9e-98f9-be8a771f90ff None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:53.220938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-0dd3ecea-0244-4a9e-98f9-be8a771f90ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:42:53.220938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:53.220938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:53.220938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:53.220938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:53.220938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:53.220938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:53.220938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:53.220938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:53.220938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:53.220938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:53.220938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:53.220938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:42:53.220938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:42:53.220938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:53.221902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:53.222954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:42:53.222954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:42:53.222954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:42:53.222954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:42:53.222954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:42:53.222954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:42:53.222954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:42:53.222954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:53.222954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 638/1271] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:42:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:53.228376 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69c69a-5af8-496b-a8b0-04281898eda7 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:53.286150 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:53.328451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69c69a-5af8-496b-a8b0-04281898eda7 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:53.361303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d492b1-ab3c-4438-8876-1c035f19dfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:53.361905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d492b1-ab3c-4438-8876-1c035f19dfeb None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:53.362192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d492b1-ab3c-4438-8876-1c035f19dfeb None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:53.362464 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d492b1-ab3c-4438-8876-1c035f19dfeb None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:53.363142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d492b1-ab3c-4438-8876-1c035f19dfeb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:53.363438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d492b1-ab3c-4438-8876-1c035f19dfeb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:53.364900 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d492b1-ab3c-4438-8876-1c035f19dfeb None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:53.376896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 634/1272] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:42:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:53.382874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3ecea-0244-4a9e-98f9-be8a771f90ff None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:53.471795 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:53.480811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd3ecea-0244-4a9e-98f9-be8a771f90ff None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:53.514502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee0f47-4ed0-431e-9b5d-1373b56c101e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:53.515148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee0f47-4ed0-431e-9b5d-1373b56c101e None admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:53.515427 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee0f47-4ed0-431e-9b5d-1373b56c101e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:53.515698 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee0f47-4ed0-431e-9b5d-1373b56c101e None admin] PATH_INFO: `/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/568189358b154b66a729fae2f0411d1b/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:53.517657 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee0f47-4ed0-431e-9b5d-1373b56c101e None admin] RBAC: Authorizing `identity:create_grant(project_id=7dee98edf19d478ca4f746ff85579fba, user_id=568189358b154b66a729fae2f0411d1b, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:53.519077 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee0f47-4ed0-431e-9b5d-1373b56c101e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:53.530947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 639/1273] 10.222.0.60 () {58 vars in 1439 bytes} [Mon Sep 19 14:42:53 2022] PUT /identity/v3/projects/7dee98edf19d478ca4f746ff85579fba/users/568189358b154b66a729fae2f0411d1b/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:42:55.691173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44729c-819f-4fde-8edc-ec0b87b8d737 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:55.691741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44729c-819f-4fde-8edc-ec0b87b8d737 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:55.691741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44729c-819f-4fde-8edc-ec0b87b8d737 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:55.693370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 635/1274] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:42:55.699084 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadc5c1-fda4-4ada-af9c-3bd28b3b9b5f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:55.699590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadc5c1-fda4-4ada-af9c-3bd28b3b9b5f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:55.699694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadc5c1-fda4-4ada-af9c-3bd28b3b9b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:55.720651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8cadc5c1-fda4-4ada-af9c-3bd28b3b9b5f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:42:55.721859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8cadc5c1-fda4-4ada-af9c-3bd28b3b9b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:42:55.786807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 640/1275] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:42:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:42:56.056285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44729c-819f-4fde-8edc-ec0b87b8d737 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:56.157773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f44729c-819f-4fde-8edc-ec0b87b8d737 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:56.175336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:56.192504 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320464c-1c19-45c6-b2ac-238f19abb0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:56.192840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320464c-1c19-45c6-b2ac-238f19abb0dd None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:56.193122 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320464c-1c19-45c6-b2ac-238f19abb0dd None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:56.193396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320464c-1c19-45c6-b2ac-238f19abb0dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:56.203754 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320464c-1c19-45c6-b2ac-238f19abb0dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-c320464c-1c19-45c6-b2ac-238f19abb0dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 19 14:42:56.208490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:56.209760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:56.210906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:56.210906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:56.210906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:56.210906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:56.210906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 19 14:42:56.210906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 19 14:42:56.210906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 19 14:42:56.210906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 19 14:42:56.210906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 19 14:42:56.210906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 19 14:42:56.210906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 19 14:42:56.210906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:42:56.210906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 636/1276] 10.222.0.60 () {56 vars in 1096 bytes} [Mon Sep 19 14:42:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:56.216232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadc5c1-fda4-4ada-af9c-3bd28b3b9b5f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:56.294658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:56.313492 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadc5c1-fda4-4ada-af9c-3bd28b3b9b5f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:56.347314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5050b4-8319-4aa8-8dd9-68d4a6293a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:56.347941 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5050b4-8319-4aa8-8dd9-68d4a6293a34 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:56.348223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5050b4-8319-4aa8-8dd9-68d4a6293a34 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:56.348496 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5050b4-8319-4aa8-8dd9-68d4a6293a34 None admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:56.349146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5050b4-8319-4aa8-8dd9-68d4a6293a34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:56.349447 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5050b4-8319-4aa8-8dd9-68d4a6293a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:56.350804 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5050b4-8319-4aa8-8dd9-68d4a6293a34 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:56.358626 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 641/1277] 10.222.0.60 () {56 vars in 1099 bytes} [Mon Sep 19 14:42:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:56.364537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320464c-1c19-45c6-b2ac-238f19abb0dd None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:56.463444 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320464c-1c19-45c6-b2ac-238f19abb0dd None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:56.496396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c27e9b-2ff4-492b-ad83-1954d640b820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:56.497010 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c27e9b-2ff4-492b-ad83-1954d640b820 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:56.497292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c27e9b-2ff4-492b-ad83-1954d640b820 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:56.497566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c27e9b-2ff4-492b-ad83-1954d640b820 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:56.507791 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c27e9b-2ff4-492b-ad83-1954d640b820 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:56.509301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c27e9b-2ff4-492b-ad83-1954d640b820 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-f7c27e9b-2ff4-492b-ad83-1954d640b820 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:42:56.523494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:56.524638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:42:56.525874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:42:56.525874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:56.525874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:42:56.525874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 637/1278] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:56.530543 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5050b4-8319-4aa8-8dd9-68d4a6293a34 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:56.632271 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5050b4-8319-4aa8-8dd9-68d4a6293a34 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:56.670112 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09175c47-184c-4f69-8eda-f8010054d322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:56.670716 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09175c47-184c-4f69-8eda-f8010054d322 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:56.670991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09175c47-184c-4f69-8eda-f8010054d322 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:56.671258 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09175c47-184c-4f69-8eda-f8010054d322 None admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:56.671912 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09175c47-184c-4f69-8eda-f8010054d322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:56.672212 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09175c47-184c-4f69-8eda-f8010054d322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:56.673567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09175c47-184c-4f69-8eda-f8010054d322 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:56.685324 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 642/1279] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:56.691059 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c27e9b-2ff4-492b-ad83-1954d640b820 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:56.721347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:56.789581 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c27e9b-2ff4-492b-ad83-1954d640b820 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:56.822598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ceb72b-4c7b-45c5-ae0c-924a87f79e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:56.823215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ceb72b-4c7b-45c5-ae0c-924a87f79e71 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:56.823497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ceb72b-4c7b-45c5-ae0c-924a87f79e71 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:56.823772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ceb72b-4c7b-45c5-ae0c-924a87f79e71 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:56.830993 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ceb72b-4c7b-45c5-ae0c-924a87f79e71 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:56.832477 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ceb72b-4c7b-45c5-ae0c-924a87f79e71 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-33ceb72b-4c7b-45c5-ae0c-924a87f79e71 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 19 14:42:56.843876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:56.845027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:56.846400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:56.846400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:56.846400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 19 14:42:56.846400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 19 14:42:56.846400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 19 14:42:56.846400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 19 14:42:56.846400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 19 14:42:56.846400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 19 14:42:56.846400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 19 14:42:56.846400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:42:56.846400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 638/1280] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:56.850764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09175c47-184c-4f69-8eda-f8010054d322 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:56.854774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:56.949043 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09175c47-184c-4f69-8eda-f8010054d322 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:56.981790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ccf42-ddbf-4c86-87f2-5f99314d7a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:56.982378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ccf42-ddbf-4c86-87f2-5f99314d7a1e None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:56.982651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ccf42-ddbf-4c86-87f2-5f99314d7a1e None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:56.982918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ccf42-ddbf-4c86-87f2-5f99314d7a1e None admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:56.983618 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ccf42-ddbf-4c86-87f2-5f99314d7a1e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:56.983942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ccf42-ddbf-4c86-87f2-5f99314d7a1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:56.985349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ccf42-ddbf-4c86-87f2-5f99314d7a1e None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:57.001470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 643/1281] 10.222.0.60 () {56 vars in 1143 bytes} [Mon Sep 19 14:42:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:57.007409 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ceb72b-4c7b-45c5-ae0c-924a87f79e71 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:57.108097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:57.118130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ceb72b-4c7b-45c5-ae0c-924a87f79e71 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:57.151377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c4f6c-2837-4f4d-bef5-5f7c433e4009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:57.152010 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c4f6c-2837-4f4d-bef5-5f7c433e4009 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:57.152291 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c4f6c-2837-4f4d-bef5-5f7c433e4009 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:57.152564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c4f6c-2837-4f4d-bef5-5f7c433e4009 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:57.217649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c4f6c-2837-4f4d-bef5-5f7c433e4009 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:57.219019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c4f6c-2837-4f4d-bef5-5f7c433e4009 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:57.304404 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:57.315119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 639/1282] 10.222.0.60 () {58 vars in 1282 bytes} [Mon Sep 19 14:42:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 19 14:42:57.321001 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ccf42-ddbf-4c86-87f2-5f99314d7a1e None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:57.382552 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:57.423232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ccf42-ddbf-4c86-87f2-5f99314d7a1e None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:57.457459 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c629e-8deb-49f4-915e-797c27859df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:57.458083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c629e-8deb-49f4-915e-797c27859df3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:57.458364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c629e-8deb-49f4-915e-797c27859df3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:57.458635 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c629e-8deb-49f4-915e-797c27859df3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:57.468442 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c629e-8deb-49f4-915e-797c27859df3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:57.469822 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c629e-8deb-49f4-915e-797c27859df3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-a85c629e-8deb-49f4-915e-797c27859df3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 19 14:42:57.485440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:57.486521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 19 14:42:57.487562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 19 14:42:57.487562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 19 14:42:57.487562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:57.487562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 644/1283] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:57.492305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c4f6c-2837-4f4d-bef5-5f7c433e4009 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:57.589010 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c4f6c-2837-4f4d-bef5-5f7c433e4009 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:57.623022 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c78b1-418a-4e5c-bdf5-73d29b22e935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:57.623877 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c78b1-418a-4e5c-bdf5-73d29b22e935 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:57.624299 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c78b1-418a-4e5c-bdf5-73d29b22e935 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:57.624578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c78b1-418a-4e5c-bdf5-73d29b22e935 None admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:57.625480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c78b1-418a-4e5c-bdf5-73d29b22e935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:57.625992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c78b1-418a-4e5c-bdf5-73d29b22e935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:57.627760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c78b1-418a-4e5c-bdf5-73d29b22e935 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:57.638779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 640/1284] 10.222.0.60 () {56 vars in 1109 bytes} [Mon Sep 19 14:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:57.644452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c629e-8deb-49f4-915e-797c27859df3 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:57.741750 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c629e-8deb-49f4-915e-797c27859df3 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:57.755482 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:57.775745 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2771d7-6e35-4901-bbb8-ad5869e2c432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:57.776390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2771d7-6e35-4901-bbb8-ad5869e2c432 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:57.776672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2771d7-6e35-4901-bbb8-ad5869e2c432 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:57.776943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2771d7-6e35-4901-bbb8-ad5869e2c432 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:57.783413 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2771d7-6e35-4901-bbb8-ad5869e2c432 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:57.784890 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2771d7-6e35-4901-bbb8-ad5869e2c432 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:57.796418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application [None req-ed2771d7-6e35-4901-bbb8-ad5869e2c432 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:42:57.796418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:42:57.796418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:42:57.796418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:42:57.796418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:42:57.796418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:42:57.796418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:42:57.796418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:42:57.796418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:42:57.796418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:42:57.796418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:42:57.796418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:42:57.796418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 19 14:42:57.796418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 19 14:42:57.796418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application with Lock( Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:42:57.797479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:42:57.798800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 19 14:42:57.798800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 19 14:42:57.798800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 19 14:42:57.798800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 19 14:42:57.798800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 19 14:42:57.798800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 19 14:42:57.798800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 19 14:42:57.798800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: ERROR keystone.server.flask.application Sep 19 14:42:57.798800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 645/1285] 10.222.0.60 () {56 vars in 1111 bytes} [Mon Sep 19 14:42:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:42:57.803808 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c78b1-418a-4e5c-bdf5-73d29b22e935 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:57.852627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:42:57.903766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c78b1-418a-4e5c-bdf5-73d29b22e935 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:57.937153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635d647-5dd8-44c6-81c1-e38822de9a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:57.937749 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635d647-5dd8-44c6-81c1-e38822de9a90 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:57.938025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635d647-5dd8-44c6-81c1-e38822de9a90 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:57.938292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635d647-5dd8-44c6-81c1-e38822de9a90 None admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:57.938959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635d647-5dd8-44c6-81c1-e38822de9a90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:57.939257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635d647-5dd8-44c6-81c1-e38822de9a90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:57.940701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635d647-5dd8-44c6-81c1-e38822de9a90 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:57.952952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 641/1286] 10.222.0.60 () {56 vars in 1148 bytes} [Mon Sep 19 14:42:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:42:57.959199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2771d7-6e35-4901-bbb8-ad5869e2c432 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:42:58.053948 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2771d7-6e35-4901-bbb8-ad5869e2c432 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:42:58.086696 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4b67b-751d-4d0b-9f95-9384f61743e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:42:58.087686 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4b67b-751d-4d0b-9f95-9384f61743e7 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:42:58.087686 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4b67b-751d-4d0b-9f95-9384f61743e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:42:58.087896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4b67b-751d-4d0b-9f95-9384f61743e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:42:58.088568 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4b67b-751d-4d0b-9f95-9384f61743e7 None admin] RBAC: Adding query filter params (role.id=d3ae59ea5d7f4bcc964c8ff473925f8e, scope.project.id=7dee98edf19d478ca4f746ff85579fba, user.id=568189358b154b66a729fae2f0411d1b) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:42:58.088871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4b67b-751d-4d0b-9f95-9384f61743e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:42:58.090286 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4b67b-751d-4d0b-9f95-9384f61743e7 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:42:58.095909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 646/1287] 10.222.0.60 () {56 vars in 1374 bytes} [Mon Sep 19 14:42:57 2022] GET /identity/v3/role_assignments?user.id=568189358b154b66a729fae2f0411d1b&scope.project.id=7dee98edf19d478ca4f746ff85579fba&role.id=d3ae59ea5d7f4bcc964c8ff473925f8e => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:43:00.231066 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d7a42-1d6a-41d5-aa52-a30ea15c7c54 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:00.231594 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d7a42-1d6a-41d5-aa52-a30ea15c7c54 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:00.231594 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d7a42-1d6a-41d5-aa52-a30ea15c7c54 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:00.232989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 642/1288] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:43:00.238203 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f97c19-1f3a-4b28-930d-73e747e17bcb None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:00.238588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f97c19-1f3a-4b28-930d-73e747e17bcb None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:00.238753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f97c19-1f3a-4b28-930d-73e747e17bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:00.257065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-82f97c19-1f3a-4b28-930d-73e747e17bcb None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:43:00.258249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-82f97c19-1f3a-4b28-930d-73e747e17bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:43:00.286998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:43:00.324584 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 647/1289] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:43:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:43:00.591077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d7a42-1d6a-41d5-aa52-a30ea15c7c54 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:43:00.701504 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d7a42-1d6a-41d5-aa52-a30ea15c7c54 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:43:00.713124 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:43:00.736774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679b8e4-2522-4778-97bf-23b8029a580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:43:00.737402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679b8e4-2522-4778-97bf-23b8029a580c None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:00.737687 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679b8e4-2522-4778-97bf-23b8029a580c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:00.737963 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679b8e4-2522-4778-97bf-23b8029a580c None admin] PATH_INFO: `/v3/services/placement` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:00.738537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679b8e4-2522-4778-97bf-23b8029a580c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:43:00.739939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679b8e4-2522-4778-97bf-23b8029a580c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-8679b8e4-2522-4778-97bf-23b8029a580c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:43:00.749092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:43:00.750334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:43:00.751401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:43:00.751401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:43:00.751401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:43:00.751401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:43:00.751401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:43:00.751401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:43:00.751401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:43:00.751401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:43:00.751401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:43:00.751401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:43:00.751401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:43:00.751401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:43:00.751401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 14:43:00.751401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:43:00.751401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 643/1290] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Sep 19 14:43:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:43:00.756191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f97c19-1f3a-4b28-930d-73e747e17bcb None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:43:00.852199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f97c19-1f3a-4b28-930d-73e747e17bcb None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:43:00.885792 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc50f0-4f8c-4d0a-9ea5-fc6c47c8a439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:43:00.886401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc50f0-4f8c-4d0a-9ea5-fc6c47c8a439 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:00.886682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc50f0-4f8c-4d0a-9ea5-fc6c47c8a439 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:00.886983 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc50f0-4f8c-4d0a-9ea5-fc6c47c8a439 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:00.887672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc50f0-4f8c-4d0a-9ea5-fc6c47c8a439 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:43:00.888027 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc50f0-4f8c-4d0a-9ea5-fc6c47c8a439 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:43:00.889398 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc50f0-4f8c-4d0a-9ea5-fc6c47c8a439 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:43:00.894381 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 648/1291] 10.222.0.60 () {56 vars in 1116 bytes} [Mon Sep 19 14:43:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:43:00.900649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679b8e4-2522-4778-97bf-23b8029a580c None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:43:00.997250 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679b8e4-2522-4778-97bf-23b8029a580c None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:43:01.029603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e07c7c-b94a-4922-a215-d950218cd047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:43:01.030212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e07c7c-b94a-4922-a215-d950218cd047 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:01.030496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e07c7c-b94a-4922-a215-d950218cd047 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:01.030771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e07c7c-b94a-4922-a215-d950218cd047 None admin] PATH_INFO: `/v3/services` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:01.031394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e07c7c-b94a-4922-a215-d950218cd047 None admin] RBAC: Adding query filter params (type=placement) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:43:01.031697 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e07c7c-b94a-4922-a215-d950218cd047 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:43:01.033106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e07c7c-b94a-4922-a215-d950218cd047 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:43:01.037454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 644/1292] 10.222.0.60 () {56 vars in 1116 bytes} [Mon Sep 19 14:43:00 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:43:03.158594 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab61e7-0771-4398-b952-e685864d9cf3 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:03.159116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab61e7-0771-4398-b952-e685864d9cf3 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:03.159116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab61e7-0771-4398-b952-e685864d9cf3 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:03.160492 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 649/1293] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:43:03.165979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe0cf8-1fde-4fcf-af16-6e094cd2e1d6 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:03.166431 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe0cf8-1fde-4fcf-af16-6e094cd2e1d6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:03.166527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe0cf8-1fde-4fcf-af16-6e094cd2e1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:03.185084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2cfe0cf8-1fde-4fcf-af16-6e094cd2e1d6 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:43:03.186255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2cfe0cf8-1fde-4fcf-af16-6e094cd2e1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:43:03.214751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:43:03.251559 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 645/1294] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:43:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:43:03.516375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab61e7-0771-4398-b952-e685864d9cf3 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:43:03.547044 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:43:03.616661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab61e7-0771-4398-b952-e685864d9cf3 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:43:03.656176 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1fd318-cc5e-4be0-96d0-9509db025b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:43:03.656792 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fd318-cc5e-4be0-96d0-9509db025b19 None admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:03.657071 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fd318-cc5e-4be0-96d0-9509db025b19 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:03.657338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fd318-cc5e-4be0-96d0-9509db025b19 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:03.657974 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1fd318-cc5e-4be0-96d0-9509db025b19 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:43:03.659452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1fd318-cc5e-4be0-96d0-9509db025b19 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:43:03.673644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 650/1295] 10.222.0.60 () {60 vars in 1141 bytes} [Mon Sep 19 14:43:03 2022] POST /identity/v3/services => generated 246 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:43:05.846351 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c855792-4ca9-423a-bbe4-8894701d0773 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:05.846937 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c855792-4ca9-423a-bbe4-8894701d0773 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:05.847199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c855792-4ca9-423a-bbe4-8894701d0773 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:05.848663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 646/1296] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:43:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:43:05.854516 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcc3ed-1a5a-47df-9f74-74ca91f90b84 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:05.854933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcc3ed-1a5a-47df-9f74-74ca91f90b84 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:05.855069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcc3ed-1a5a-47df-9f74-74ca91f90b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:05.874336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-87dcc3ed-1a5a-47df-9f74-74ca91f90b84 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:43:05.875530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-87dcc3ed-1a5a-47df-9f74-74ca91f90b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:43:05.947751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 651/1297] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:43:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 19 14:43:06.221966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c855792-4ca9-423a-bbe4-8894701d0773 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:43:06.319166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c855792-4ca9-423a-bbe4-8894701d0773 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:43:06.352909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc215c-98ff-4f73-9522-54c43c7e5f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:43:06.353519 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc215c-98ff-4f73-9522-54c43c7e5f8f None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:06.353795 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc215c-98ff-4f73-9522-54c43c7e5f8f None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:06.354061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc215c-98ff-4f73-9522-54c43c7e5f8f None admin] PATH_INFO: `/v3/services/placement` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:06.354620 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc215c-98ff-4f73-9522-54c43c7e5f8f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:43:06.356006 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc215c-98ff-4f73-9522-54c43c7e5f8f None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-95cc215c-98ff-4f73-9522-54c43c7e5f8f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:43:06.364515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:43:06.365774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:43:06.366728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:43:06.366728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:43:06.366728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:43:06.366728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:43:06.366728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:43:06.366728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:43:06.366728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:43:06.366728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:43:06.366728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:43:06.366728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:43:06.366728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:43:06.366728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:43:06.366728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 14:43:06.366728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:43:06.366728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 647/1298] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Sep 19 14:43:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:43:06.372173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcc3ed-1a5a-47df-9f74-74ca91f90b84 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:43:06.390472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:43:06.480675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcc3ed-1a5a-47df-9f74-74ca91f90b84 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:43:06.514199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5a960-3935-4e97-a6a1-21e518a807c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:43:06.514804 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5a960-3935-4e97-a6a1-21e518a807c3 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:06.515083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5a960-3935-4e97-a6a1-21e518a807c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:06.515353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5a960-3935-4e97-a6a1-21e518a807c3 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:06.516000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5a960-3935-4e97-a6a1-21e518a807c3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:43:06.516301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5a960-3935-4e97-a6a1-21e518a807c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:43:06.517663 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5a960-3935-4e97-a6a1-21e518a807c3 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:43:06.522578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 652/1299] 10.222.0.60 () {56 vars in 1116 bytes} [Mon Sep 19 14:43:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:43:06.528756 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc215c-98ff-4f73-9522-54c43c7e5f8f None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:43:06.560672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:43:06.628347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc215c-98ff-4f73-9522-54c43c7e5f8f None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:43:06.668982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a7530-179c-4dd5-8d83-38af88d50e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:43:06.669656 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a7530-179c-4dd5-8d83-38af88d50e2b None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:06.669982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a7530-179c-4dd5-8d83-38af88d50e2b None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:06.670382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a7530-179c-4dd5-8d83-38af88d50e2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:06.671087 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a7530-179c-4dd5-8d83-38af88d50e2b None admin] RBAC: Adding query filter params (interface=public, service_id=517ffc3b265844f4ac9db058e1053174, region_id=RegionOne) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:43:06.671419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a7530-179c-4dd5-8d83-38af88d50e2b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:43:06.673006 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a7530-179c-4dd5-8d83-38af88d50e2b None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:43:06.683884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 648/1300] 10.222.0.60 () {56 vars in 1251 bytes} [Mon Sep 19 14:43:06 2022] GET /identity/v3/endpoints?service_id=517ffc3b265844f4ac9db058e1053174&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:43:06.685525 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5a960-3935-4e97-a6a1-21e518a807c3 None admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:43:06.769459 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:43:06.787916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5a960-3935-4e97-a6a1-21e518a807c3 None admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:43:06.826526 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63baddd4-ae26-455b-8e1b-a9be2de5cc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:43:06.827130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63baddd4-ae26-455b-8e1b-a9be2de5cc2d None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:06.827407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63baddd4-ae26-455b-8e1b-a9be2de5cc2d None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:06.827677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63baddd4-ae26-455b-8e1b-a9be2de5cc2d None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:06.828304 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63baddd4-ae26-455b-8e1b-a9be2de5cc2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:43:06.829790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63baddd4-ae26-455b-8e1b-a9be2de5cc2d None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:43:06.837943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 653/1301] 10.222.0.60 () {56 vars in 1087 bytes} [Mon Sep 19 14:43:06 2022] GET /identity/v3/services => generated 2191 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:43:08.966417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0533326-f829-4e05-86fb-f4f9a53e8401 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:08.966995 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0533326-f829-4e05-86fb-f4f9a53e8401 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:08.966995 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0533326-f829-4e05-86fb-f4f9a53e8401 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:08.968332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 649/1302] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:43:08.973914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12f5a2-3082-4caf-b1b2-f02e45564f63 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:08.974351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12f5a2-3082-4caf-b1b2-f02e45564f63 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:08.974463 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12f5a2-3082-4caf-b1b2-f02e45564f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:08.992350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-cb12f5a2-3082-4caf-b1b2-f02e45564f63 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:43:08.993536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cb12f5a2-3082-4caf-b1b2-f02e45564f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:43:09.058564 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 654/1303] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:43:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 19 14:43:09.326773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0533326-f829-4e05-86fb-f4f9a53e8401 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:43:09.419868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:43:09.426961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0533326-f829-4e05-86fb-f4f9a53e8401 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:43:09.460720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38ac70-7d6e-403a-9a99-1a07ee65e7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:43:09.461340 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38ac70-7d6e-403a-9a99-1a07ee65e7e3 None admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:09.461621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38ac70-7d6e-403a-9a99-1a07ee65e7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:09.461895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38ac70-7d6e-403a-9a99-1a07ee65e7e3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:09.462498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e38ac70-7d6e-403a-9a99-1a07ee65e7e3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:43:09.463945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e38ac70-7d6e-403a-9a99-1a07ee65e7e3 None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application [None req-3e38ac70-7d6e-403a-9a99-1a07ee65e7e3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 19 14:43:09.472621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 19 14:43:09.474078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.get_or_create( Sep 19 14:43:09.475216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 19 14:43:09.475216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application with Lock( Sep 19 14:43:09.475216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 19 14:43:09.475216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self._enter() Sep 19 14:43:09.475216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 19 14:43:09.475216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 19 14:43:09.475216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 19 14:43:09.475216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application return self.creator() Sep 19 14:43:09.475216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 19 14:43:09.475216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application created_value = creator( Sep 19 14:43:09.475216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 19 14:43:09.475216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 19 14:43:09.475216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 19 14:43:09.475216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: ERROR keystone.server.flask.application Sep 19 14:43:09.475216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 650/1304] 10.222.0.60 () {56 vars in 1117 bytes} [Mon Sep 19 14:43:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 19 14:43:09.480304 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12f5a2-3082-4caf-b1b2-f02e45564f63 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:43:09.553482 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:43:09.577360 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12f5a2-3082-4caf-b1b2-f02e45564f63 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:43:09.610075 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3f14a-41d3-4705-b7e2-1b57f29a4172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:43:09.610680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3f14a-41d3-4705-b7e2-1b57f29a4172 None admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:09.610957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3f14a-41d3-4705-b7e2-1b57f29a4172 None admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:09.611226 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3f14a-41d3-4705-b7e2-1b57f29a4172 None admin] PATH_INFO: `/v3/services` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:09.611880 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3f14a-41d3-4705-b7e2-1b57f29a4172 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:43:09.612251 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3f14a-41d3-4705-b7e2-1b57f29a4172 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:43:09.613596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3f14a-41d3-4705-b7e2-1b57f29a4172 None admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:43:09.618543 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 655/1305] 10.222.0.60 () {56 vars in 1116 bytes} [Mon Sep 19 14:43:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:43:09.624423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38ac70-7d6e-403a-9a99-1a07ee65e7e3 None admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:43:09.725794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38ac70-7d6e-403a-9a99-1a07ee65e7e3 None admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:43:09.758825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202733c9-9a8e-4a90-925b-732af2b16f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:43:09.759979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202733c9-9a8e-4a90-925b-732af2b16f6c None admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:43:09.760263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202733c9-9a8e-4a90-925b-732af2b16f6c None admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:43:09.760536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202733c9-9a8e-4a90-925b-732af2b16f6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:43:09.761192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202733c9-9a8e-4a90-925b-732af2b16f6c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:43:09.762558 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202733c9-9a8e-4a90-925b-732af2b16f6c None admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:43:09.781948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 651/1306] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:43:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:43:19.615614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=30 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:43:19.777836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:14.478738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f6f2f-fba8-4270-8be4-7e9353f80e14 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:14.479219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f6f2f-fba8-4270-8be4-7e9353f80e14 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:14.479387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f6f2f-fba8-4270-8be4-7e9353f80e14 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:14.481154 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 656/1307] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:44:14.486918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1297b-d201-4285-b65d-1df9bec76630 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:14.487751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1297b-d201-4285-b65d-1df9bec76630 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:14.488216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1297b-d201-4285-b65d-1df9bec76630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:14.583305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9de1297b-d201-4285-b65d-1df9bec76630 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:44:14.584564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9de1297b-d201-4285-b65d-1df9bec76630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:14.726479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 652/1308] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:44:15.022533 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cb837-d57f-432e-a0b5-fa9fa76d1645 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:15.023355 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cb837-d57f-432e-a0b5-fa9fa76d1645 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:15.023625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39cb837-d57f-432e-a0b5-fa9fa76d1645 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:15.025118 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 657/1309] 10.222.0.60 () {54 vars in 963 bytes} [Mon Sep 19 14:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:44:15.031556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd18217-ebf6-479e-aeb8-2760e3deb68f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:15.031868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd18217-ebf6-479e-aeb8-2760e3deb68f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:15.032300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd18217-ebf6-479e-aeb8-2760e3deb68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:15.116535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-dfd18217-ebf6-479e-aeb8-2760e3deb68f None None] MFA Rules not processed for user `2fbef7b24f134f5eb1193aed1b7ee69c`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:44:15.118019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dfd18217-ebf6-479e-aeb8-2760e3deb68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:15.140983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:15.201343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 653/1310] 10.222.0.60 () {58 vars in 1060 bytes} [Mon Sep 19 14:44:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:44:15.210806 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39cb837-d57f-432e-a0b5-fa9fa76d1645 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:44:15.212790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a39cb837-d57f-432e-a0b5-fa9fa76d1645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:15.345748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39cb837-d57f-432e-a0b5-fa9fa76d1645 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:44:15.431478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca7f485-a98e-400d-9bbb-cad6fae45518 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:44:15.432151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7f485-a98e-400d-9bbb-cad6fae45518 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:15.432433 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7f485-a98e-400d-9bbb-cad6fae45518 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:15.432882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca7f485-a98e-400d-9bbb-cad6fae45518 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:15.434490 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-aca7f485-a98e-400d-9bbb-cad6fae45518 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:15.496924 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:15.560103 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7f485-a98e-400d-9bbb-cad6fae45518 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:44:15.561504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca7f485-a98e-400d-9bbb-cad6fae45518 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:44:15.582099 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 658/1311] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:44:18.125332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f6f63-3fa7-447e-a458-4e5280b7dbff None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:18.125855 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f6f63-3fa7-447e-a458-4e5280b7dbff None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:18.125855 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f6f63-3fa7-447e-a458-4e5280b7dbff None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:18.127123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 654/1312] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:44:18.132279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad61c7c-0911-48bd-af6f-09762111732c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:18.132666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad61c7c-0911-48bd-af6f-09762111732c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:18.132835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad61c7c-0911-48bd-af6f-09762111732c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:18.214382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8ad61c7c-0911-48bd-af6f-09762111732c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:44:18.215599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8ad61c7c-0911-48bd-af6f-09762111732c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:18.320372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 659/1313] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:44:18.607253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f6f63-3fa7-447e-a458-4e5280b7dbff None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:44:18.608827 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-257f6f63-3fa7-447e-a458-4e5280b7dbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:18.713952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f6f63-3fa7-447e-a458-4e5280b7dbff None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:44:18.785813 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:18.788371 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd5f4f-c9a9-4ce6-a657-bd4bdabe1043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:44:18.789248 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd5f4f-c9a9-4ce6-a657-bd4bdabe1043 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:18.789704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd5f4f-c9a9-4ce6-a657-bd4bdabe1043 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:18.790024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd5f4f-c9a9-4ce6-a657-bd4bdabe1043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:18.791845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-51fd5f4f-c9a9-4ce6-a657-bd4bdabe1043 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:18.901263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd5f4f-c9a9-4ce6-a657-bd4bdabe1043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:44:18.902707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd5f4f-c9a9-4ce6-a657-bd4bdabe1043 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:44:18.920639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 655/1314] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:44:21.384345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffaab4c-c9d4-41bf-b1b4-24f9b2378e16 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:21.384878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffaab4c-c9d4-41bf-b1b4-24f9b2378e16 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:21.384878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffaab4c-c9d4-41bf-b1b4-24f9b2378e16 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:21.386134 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 660/1315] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:44:21.391438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f11645-517f-4d53-be92-a968f033ff2e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:21.391891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f11645-517f-4d53-be92-a968f033ff2e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:21.391991 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f11645-517f-4d53-be92-a968f033ff2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:21.456458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-64f11645-517f-4d53-be92-a968f033ff2e None None] MFA Rules not processed for user `89fc893d73ba473db202a90d053d6fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:44:21.457707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-64f11645-517f-4d53-be92-a968f033ff2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:21.508727 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:21.552200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 656/1316] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:44:21 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:44:21.848393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffaab4c-c9d4-41bf-b1b4-24f9b2378e16 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:44:21.849937 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3ffaab4c-c9d4-41bf-b1b4-24f9b2378e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:21.859625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:21.957438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffaab4c-c9d4-41bf-b1b4-24f9b2378e16 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:44:22.031283 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee184e-eb9a-44cb-94df-d38aa92a1632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:44:22.031917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee184e-eb9a-44cb-94df-d38aa92a1632 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:22.032192 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee184e-eb9a-44cb-94df-d38aa92a1632 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:22.032465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee184e-eb9a-44cb-94df-d38aa92a1632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:22.034052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-59ee184e-eb9a-44cb-94df-d38aa92a1632 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:22.096834 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:22.145697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee184e-eb9a-44cb-94df-d38aa92a1632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:44:22.147128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee184e-eb9a-44cb-94df-d38aa92a1632 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:44:22.164938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 661/1317] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:44:21 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:44:24.864494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3193f00-83a6-4447-9f4b-1ad0faeef66d None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:24.865040 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3193f00-83a6-4447-9f4b-1ad0faeef66d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:24.865040 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3193f00-83a6-4447-9f4b-1ad0faeef66d None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:24.866885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 657/1318] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:44:24.872408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53bf4e-08c4-4270-90c4-eac6abd3c6be None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:24.872817 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53bf4e-08c4-4270-90c4-eac6abd3c6be None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:24.872982 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53bf4e-08c4-4270-90c4-eac6abd3c6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:24.940724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-7e53bf4e-08c4-4270-90c4-eac6abd3c6be None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:44:24.942089 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7e53bf4e-08c4-4270-90c4-eac6abd3c6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:25.036394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 662/1319] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:44:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:44:25.333691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3193f00-83a6-4447-9f4b-1ad0faeef66d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:44:25.335272 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f3193f00-83a6-4447-9f4b-1ad0faeef66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:25.442840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3193f00-83a6-4447-9f4b-1ad0faeef66d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:44:25.524337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be072e-2a95-476e-af1c-11914107f79a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:44:25.524941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be072e-2a95-476e-af1c-11914107f79a service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:25.525221 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be072e-2a95-476e-af1c-11914107f79a service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:25.525490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be072e-2a95-476e-af1c-11914107f79a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:25.527091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-83be072e-2a95-476e-af1c-11914107f79a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:25.533994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:25.637054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be072e-2a95-476e-af1c-11914107f79a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:44:25.638616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be072e-2a95-476e-af1c-11914107f79a service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:44:25.658957 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 658/1320] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:44:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:44:28.483572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9e89a-532d-45df-9772-8464763a10bd None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:28.484142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9e89a-532d-45df-9772-8464763a10bd None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:28.484142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9e89a-532d-45df-9772-8464763a10bd None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:28.485461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 663/1321] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:44:28.491452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6e724-1215-4665-9bdd-ad9547a37b6c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:28.491939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6e724-1215-4665-9bdd-ad9547a37b6c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:28.492067 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6e724-1215-4665-9bdd-ad9547a37b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:28.566994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5ee6e724-1215-4665-9bdd-ad9547a37b6c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:44:28.568170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5ee6e724-1215-4665-9bdd-ad9547a37b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:28.629304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:28.680310 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 659/1322] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:44:28.964525 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9e89a-532d-45df-9772-8464763a10bd None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:44:28.966100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d4a9e89a-532d-45df-9772-8464763a10bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:28.987714 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:29.076131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9e89a-532d-45df-9772-8464763a10bd None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:44:29.158887 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa0c85-0697-490a-8d9d-3c3c855480a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:44:29.159502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa0c85-0697-490a-8d9d-3c3c855480a8 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:29.159784 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa0c85-0697-490a-8d9d-3c3c855480a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:29.160092 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa0c85-0697-490a-8d9d-3c3c855480a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:29.161689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4bfa0c85-0697-490a-8d9d-3c3c855480a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:29.234809 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:29.277553 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa0c85-0697-490a-8d9d-3c3c855480a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:44:29.278833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa0c85-0697-490a-8d9d-3c3c855480a8 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:44:29.298904 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 664/1323] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:44:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:44:31.519586 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38eb1c4-f46e-4410-bcd9-f29ad1dbedb0 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:31.520175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38eb1c4-f46e-4410-bcd9-f29ad1dbedb0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:31.520175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38eb1c4-f46e-4410-bcd9-f29ad1dbedb0 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:31.521655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 660/1324] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:44:31.526800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d659a6-0bad-4918-9a6a-55a281e92459 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:31.527235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d659a6-0bad-4918-9a6a-55a281e92459 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:31.527354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d659a6-0bad-4918-9a6a-55a281e92459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:31.593037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a1d659a6-0bad-4918-9a6a-55a281e92459 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:44:31.594234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a1d659a6-0bad-4918-9a6a-55a281e92459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:31.702874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 665/1325] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:44:32.008799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05403f5b-c7c9-4f10-9c1d-600844225c84 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:32.009224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05403f5b-c7c9-4f10-9c1d-600844225c84 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:32.009568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05403f5b-c7c9-4f10-9c1d-600844225c84 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:32.010969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 661/1326] 10.222.0.60 () {54 vars in 963 bytes} [Mon Sep 19 14:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:44:32.017008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389a1ef-9368-48d4-b217-f17b6ea1699f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:32.017366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389a1ef-9368-48d4-b217-f17b6ea1699f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:32.017648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389a1ef-9368-48d4-b217-f17b6ea1699f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:32.049412 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:32.097794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8389a1ef-9368-48d4-b217-f17b6ea1699f None None] MFA Rules not processed for user `2fbef7b24f134f5eb1193aed1b7ee69c`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:44:32.099010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8389a1ef-9368-48d4-b217-f17b6ea1699f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:32.175573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 666/1327] 10.222.0.60 () {58 vars in 1060 bytes} [Mon Sep 19 14:44:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:44:32.185046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05403f5b-c7c9-4f10-9c1d-600844225c84 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:44:32.186588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-05403f5b-c7c9-4f10-9c1d-600844225c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:32.307846 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05403f5b-c7c9-4f10-9c1d-600844225c84 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:44:32.385789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef672462-baaf-4f8a-a9b1-739777e6d7f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:44:32.386407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef672462-baaf-4f8a-a9b1-739777e6d7f9 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:32.386686 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef672462-baaf-4f8a-a9b1-739777e6d7f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:32.386954 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef672462-baaf-4f8a-a9b1-739777e6d7f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:32.388590 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ef672462-baaf-4f8a-a9b1-739777e6d7f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:32.405883 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:32.497766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef672462-baaf-4f8a-a9b1-739777e6d7f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:44:32.499118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef672462-baaf-4f8a-a9b1-739777e6d7f9 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:44:32.517365 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 662/1328] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:44:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:44:35.005175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7742a87-1637-484c-a078-1c20660f3d7a None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:35.005636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7742a87-1637-484c-a078-1c20660f3d7a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:35.005752 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7742a87-1637-484c-a078-1c20660f3d7a None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:35.007949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 667/1329] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:44:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:44:35.013472 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cb072-111b-4955-bf3c-80d2aa49343d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:35.013966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cb072-111b-4955-bf3c-80d2aa49343d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:35.014060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9cb072-111b-4955-bf3c-80d2aa49343d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:35.086568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-df9cb072-111b-4955-bf3c-80d2aa49343d None None] MFA Rules not processed for user `89fc893d73ba473db202a90d053d6fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:44:35.088449 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-df9cb072-111b-4955-bf3c-80d2aa49343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:35.156008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:35.189230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 663/1330] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:44:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:44:35.474772 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7742a87-1637-484c-a078-1c20660f3d7a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:44:35.476907 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c7742a87-1637-484c-a078-1c20660f3d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:35.580585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7742a87-1637-484c-a078-1c20660f3d7a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:44:35.609485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:35.668162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67d65e-9b4c-4669-9132-b0b249e6acb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:44:35.668774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67d65e-9b4c-4669-9132-b0b249e6acb1 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:35.669053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67d65e-9b4c-4669-9132-b0b249e6acb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:35.669321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67d65e-9b4c-4669-9132-b0b249e6acb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:35.670950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4d67d65e-9b4c-4669-9132-b0b249e6acb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:35.786816 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67d65e-9b4c-4669-9132-b0b249e6acb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:44:35.788245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67d65e-9b4c-4669-9132-b0b249e6acb1 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:44:35.807037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 668/1331] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:44:35 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:44:38.400823 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f2083-bde5-4e26-863a-b03930f8d896 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:38.401403 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f2083-bde5-4e26-863a-b03930f8d896 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:38.401403 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f2083-bde5-4e26-863a-b03930f8d896 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:38.402720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 664/1332] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:44:38.408119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8674e9-ba8b-4cd6-9253-39d9470a3817 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:38.408540 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8674e9-ba8b-4cd6-9253-39d9470a3817 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:38.408657 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8674e9-ba8b-4cd6-9253-39d9470a3817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:38.471498 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:38.474409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-cb8674e9-ba8b-4cd6-9253-39d9470a3817 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:44:38.476913 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cb8674e9-ba8b-4cd6-9253-39d9470a3817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:38.582324 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 669/1333] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:44:38.866322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f2083-bde5-4e26-863a-b03930f8d896 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:44:38.867895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-681f2083-bde5-4e26-863a-b03930f8d896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:38.969593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f2083-bde5-4e26-863a-b03930f8d896 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:44:39.046107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc446f8c-79e0-42c9-b63a-024b1db9f273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:44:39.046713 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc446f8c-79e0-42c9-b63a-024b1db9f273 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:39.046998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc446f8c-79e0-42c9-b63a-024b1db9f273 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:39.047270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc446f8c-79e0-42c9-b63a-024b1db9f273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:39.048904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bc446f8c-79e0-42c9-b63a-024b1db9f273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:39.076488 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:39.159410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc446f8c-79e0-42c9-b63a-024b1db9f273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:44:39.160881 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc446f8c-79e0-42c9-b63a-024b1db9f273 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:44:39.179187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 665/1334] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:44:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:44:41.490130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd633f74-fb12-4ebc-9d45-9b5b09e669f6 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:41.490728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd633f74-fb12-4ebc-9d45-9b5b09e669f6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:41.490728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd633f74-fb12-4ebc-9d45-9b5b09e669f6 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:41.492123 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 670/1335] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:44:41.497819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35eeb8-0886-4f3d-9357-80a8d809b036 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:41.498282 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35eeb8-0886-4f3d-9357-80a8d809b036 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:41.498413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35eeb8-0886-4f3d-9357-80a8d809b036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:41.564967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-cc35eeb8-0886-4f3d-9357-80a8d809b036 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:44:41.566172 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cc35eeb8-0886-4f3d-9357-80a8d809b036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:41.639891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:41.663207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 666/1336] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:44:41.953073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd633f74-fb12-4ebc-9d45-9b5b09e669f6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:44:41.954615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bd633f74-fb12-4ebc-9d45-9b5b09e669f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:42.056988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd633f74-fb12-4ebc-9d45-9b5b09e669f6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:44:42.087289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:42.134596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554148d0-2c31-4ab7-8090-56a118e900fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:44:42.135464 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554148d0-2c31-4ab7-8090-56a118e900fa service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:42.135932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554148d0-2c31-4ab7-8090-56a118e900fa service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:42.136426 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554148d0-2c31-4ab7-8090-56a118e900fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:42.138780 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-554148d0-2c31-4ab7-8090-56a118e900fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:42.247127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554148d0-2c31-4ab7-8090-56a118e900fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:44:42.248582 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554148d0-2c31-4ab7-8090-56a118e900fa service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:44:42.267380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 671/1337] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:44:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:44:44.943771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349956b-db6e-4949-a9bf-21d2eaa83a0e None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:44.944382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349956b-db6e-4949-a9bf-21d2eaa83a0e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:44.944382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349956b-db6e-4949-a9bf-21d2eaa83a0e None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:44.945583 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 667/1338] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:44:44.951137 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6f4a1-a782-4430-a689-6c6782c9b3ae None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:44.951626 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6f4a1-a782-4430-a689-6c6782c9b3ae None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:44.951724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6f4a1-a782-4430-a689-6c6782c9b3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:45.016260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-fdb6f4a1-a782-4430-a689-6c6782c9b3ae None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:44:45.017465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fdb6f4a1-a782-4430-a689-6c6782c9b3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:45.025913 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:45.115110 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 672/1339] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:44:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:44:45.401298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349956b-db6e-4949-a9bf-21d2eaa83a0e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:44:45.402838 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7349956b-db6e-4949-a9bf-21d2eaa83a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:45.505224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349956b-db6e-4949-a9bf-21d2eaa83a0e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:44:45.577831 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ca0fd-218a-45a2-9690-51703dec0989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:44:45.578435 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ca0fd-218a-45a2-9690-51703dec0989 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:45.578715 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ca0fd-218a-45a2-9690-51703dec0989 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:45.578985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ca0fd-218a-45a2-9690-51703dec0989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:45.580670 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-714ca0fd-218a-45a2-9690-51703dec0989 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:45.615362 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:45.690772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ca0fd-218a-45a2-9690-51703dec0989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:44:45.692250 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ca0fd-218a-45a2-9690-51703dec0989 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:44:45.710954 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 668/1340] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:44:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:44:49.823146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08409428-b621-4895-9fe3-d63f1c04dc97 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:49.823706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08409428-b621-4895-9fe3-d63f1c04dc97 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:49.823706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08409428-b621-4895-9fe3-d63f1c04dc97 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:49.825052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 673/1341] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:44:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:44:49.832016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2a530-cc76-4a40-a9b4-2b593629f2f4 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:49.832843 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2a530-cc76-4a40-a9b4-2b593629f2f4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:49.833253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2a530-cc76-4a40-a9b4-2b593629f2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:49.907160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c0c2a530-cc76-4a40-a9b4-2b593629f2f4 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:44:49.908470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c0c2a530-cc76-4a40-a9b4-2b593629f2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:50.002958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:50.015444 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 669/1342] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:44:50.300866 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08409428-b621-4895-9fe3-d63f1c04dc97 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:44:50.302460 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-08409428-b621-4895-9fe3-d63f1c04dc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:50.408347 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08409428-b621-4895-9fe3-d63f1c04dc97 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:44:50.448062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:50.485993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0e60f-1c6b-4c49-b2de-898704d55464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:44:50.486612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0e60f-1c6b-4c49-b2de-898704d55464 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:50.486892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0e60f-1c6b-4c49-b2de-898704d55464 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:50.487166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0e60f-1c6b-4c49-b2de-898704d55464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:50.488793 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1be0e60f-1c6b-4c49-b2de-898704d55464 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:50.598192 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0e60f-1c6b-4c49-b2de-898704d55464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:44:50.599585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0e60f-1c6b-4c49-b2de-898704d55464 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:44:50.617975 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 674/1343] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:44:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:44:53.374851 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d73e5d-5c2e-4585-a4b5-00e103a8765f None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:53.375368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d73e5d-5c2e-4585-a4b5-00e103a8765f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:53.375368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d73e5d-5c2e-4585-a4b5-00e103a8765f None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:53.376736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 670/1344] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:44:53.381963 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec1438-c259-477b-894d-abda8c2214c7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:53.382379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec1438-c259-477b-894d-abda8c2214c7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:53.382648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec1438-c259-477b-894d-abda8c2214c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:53.449607 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5fec1438-c259-477b-894d-abda8c2214c7 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:44:53.450851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5fec1438-c259-477b-894d-abda8c2214c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:53.469552 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:53.553083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 675/1345] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:44:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:44:53.845889 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d73e5d-5c2e-4585-a4b5-00e103a8765f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:44:53.847422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-88d73e5d-5c2e-4585-a4b5-00e103a8765f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:53.957144 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d73e5d-5c2e-4585-a4b5-00e103a8765f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:44:54.040051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea857c12-da4f-46f3-9882-bae1316b4948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:44:54.040665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea857c12-da4f-46f3-9882-bae1316b4948 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:54.040945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea857c12-da4f-46f3-9882-bae1316b4948 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:54.041373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea857c12-da4f-46f3-9882-bae1316b4948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:54.043165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ea857c12-da4f-46f3-9882-bae1316b4948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:54.091379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:54.160895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea857c12-da4f-46f3-9882-bae1316b4948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:44:54.163125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea857c12-da4f-46f3-9882-bae1316b4948 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:44:54.182416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 671/1346] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:44:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:44:58.512314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184a894-21af-4a1b-8297-ff64516ae6a3 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:58.512840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184a894-21af-4a1b-8297-ff64516ae6a3 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:58.512936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184a894-21af-4a1b-8297-ff64516ae6a3 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:58.514210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 676/1347] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:44:58.520228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0fa89-6862-4506-b9f0-83739649c00e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:58.520811 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0fa89-6862-4506-b9f0-83739649c00e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:58.520811 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0fa89-6862-4506-b9f0-83739649c00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:58.593364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-75f0fa89-6862-4506-b9f0-83739649c00e None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:44:58.594575 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-75f0fa89-6862-4506-b9f0-83739649c00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:58.691499 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:58.694949 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 672/1348] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:44:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:44:58.976631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184a894-21af-4a1b-8297-ff64516ae6a3 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:44:58.978166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c184a894-21af-4a1b-8297-ff64516ae6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:59.083333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184a894-21af-4a1b-8297-ff64516ae6a3 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:44:59.133043 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:44:59.159778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20092e26-240f-4272-b22b-5736b203717b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:44:59.160411 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20092e26-240f-4272-b22b-5736b203717b service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:44:59.160689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20092e26-240f-4272-b22b-5736b203717b service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:44:59.160957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20092e26-240f-4272-b22b-5736b203717b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:44:59.163075 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-20092e26-240f-4272-b22b-5736b203717b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:44:59.273822 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20092e26-240f-4272-b22b-5736b203717b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:44:59.275169 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20092e26-240f-4272-b22b-5736b203717b service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:44:59.293350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 677/1349] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:44:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:45:04.267054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e664aa-c1ed-44d2-b2e9-8d34f757aaff None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:04.267759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e664aa-c1ed-44d2-b2e9-8d34f757aaff None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:04.267759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e664aa-c1ed-44d2-b2e9-8d34f757aaff None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:04.270121 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 673/1350] 10.222.0.60 () {54 vars in 921 bytes} [Mon Sep 19 14:45:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:04.275956 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf67922-f48f-4895-8e6f-15b5f986e7a0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:04.276426 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf67922-f48f-4895-8e6f-15b5f986e7a0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:04.277390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf67922-f48f-4895-8e6f-15b5f986e7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:04.384682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-abf67922-f48f-4895-8e6f-15b5f986e7a0 None None] MFA Rules not processed for user `568189358b154b66a729fae2f0411d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:04.386223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-abf67922-f48f-4895-8e6f-15b5f986e7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:04.410155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:04.522304 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 678/1351] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Sep 19 14:45:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:08.821504 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9156073-2bb2-4e64-a90e-88f9258be08e None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:08.822103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9156073-2bb2-4e64-a90e-88f9258be08e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:08.822103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9156073-2bb2-4e64-a90e-88f9258be08e None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:08.823381 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 674/1352] 10.222.0.60 () {54 vars in 921 bytes} [Mon Sep 19 14:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:08.828932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128a271-cefc-443f-bbdd-6b26a9f3b75f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:08.829479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128a271-cefc-443f-bbdd-6b26a9f3b75f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:08.829479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128a271-cefc-443f-bbdd-6b26a9f3b75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:08.914920 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e128a271-cefc-443f-bbdd-6b26a9f3b75f None None] MFA Rules not processed for user `568189358b154b66a729fae2f0411d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:08.916606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e128a271-cefc-443f-bbdd-6b26a9f3b75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:08.991960 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:09.034427 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 679/1353] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Sep 19 14:45:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:10.339127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1f8d9-2637-40dd-a8d1-27b6b855f3e9 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:10.339642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1f8d9-2637-40dd-a8d1-27b6b855f3e9 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:10.339642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1f8d9-2637-40dd-a8d1-27b6b855f3e9 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:10.340961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 675/1354] 10.222.0.60 () {54 vars in 921 bytes} [Mon Sep 19 14:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:10.346209 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9c2c9-0d99-40b5-b2b6-eb2b2bc92ff0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:10.346632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9c2c9-0d99-40b5-b2b6-eb2b2bc92ff0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:10.346889 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9c2c9-0d99-40b5-b2b6-eb2b2bc92ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:10.422984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0dd9c2c9-0d99-40b5-b2b6-eb2b2bc92ff0 None None] MFA Rules not processed for user `568189358b154b66a729fae2f0411d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:10.424210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0dd9c2c9-0d99-40b5-b2b6-eb2b2bc92ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:10.532635 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 680/1355] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Sep 19 14:45:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:45:14.165317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f618bc9-0bed-4a49-9954-d6c45d92e221 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:14.166142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f618bc9-0bed-4a49-9954-d6c45d92e221 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:14.166591 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f618bc9-0bed-4a49-9954-d6c45d92e221 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:14.168352 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 676/1356] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:45:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:14.174038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5da11-eed6-4512-b56c-a5fc3cbab792 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:14.174541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5da11-eed6-4512-b56c-a5fc3cbab792 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:14.174782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5da11-eed6-4512-b56c-a5fc3cbab792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:14.212703 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:14.247267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f8d5da11-eed6-4512-b56c-a5fc3cbab792 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:14.248701 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f8d5da11-eed6-4512-b56c-a5fc3cbab792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:14.355726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 681/1357] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:14.661677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c02eb8-a59b-4411-8d2f-4a7b92ce2d8f None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:14.661946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c02eb8-a59b-4411-8d2f-4a7b92ce2d8f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:14.662505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c02eb8-a59b-4411-8d2f-4a7b92ce2d8f None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:14.664276 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 677/1358] 10.222.0.60 () {54 vars in 965 bytes} [Mon Sep 19 14:45:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:14.685979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8de95-8908-48ca-a9a8-275f36d50616 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:14.686726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8de95-8908-48ca-a9a8-275f36d50616 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:14.687512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8de95-8908-48ca-a9a8-275f36d50616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:14.771622 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-99e8de95-8908-48ca-a9a8-275f36d50616 None None] MFA Rules not processed for user `59b5d2fd362b417c9b9be75f88a667ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:14.773061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-99e8de95-8908-48ca-a9a8-275f36d50616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:14.827165 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:14.904731 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 682/1359] 10.222.0.60 () {58 vars in 1062 bytes} [Mon Sep 19 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:14.912873 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c02eb8-a59b-4411-8d2f-4a7b92ce2d8f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:45:14.914368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b5c02eb8-a59b-4411-8d2f-4a7b92ce2d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:15.080639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c02eb8-a59b-4411-8d2f-4a7b92ce2d8f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:45:15.162443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:15.207194 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7b66e-0b31-4185-af25-8f7b33c30af1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:45:15.208127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7b66e-0b31-4185-af25-8f7b33c30af1 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:15.208221 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7b66e-0b31-4185-af25-8f7b33c30af1 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:15.208451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7b66e-0b31-4185-af25-8f7b33c30af1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:15.210133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dbd7b66e-0b31-4185-af25-8f7b33c30af1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:15.311920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7b66e-0b31-4185-af25-8f7b33c30af1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:45:15.313544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7b66e-0b31-4185-af25-8f7b33c30af1 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:45:15.336228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 678/1360] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:45:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:45:17.775761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fba67-7eaa-4b1f-af59-75d3174c2dd7 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:17.776315 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fba67-7eaa-4b1f-af59-75d3174c2dd7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:17.776444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fba67-7eaa-4b1f-af59-75d3174c2dd7 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:17.777729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 683/1361] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:17.785526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734a268-9f22-4fac-836b-88a3a0decc01 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:17.785961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734a268-9f22-4fac-836b-88a3a0decc01 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:17.786109 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734a268-9f22-4fac-836b-88a3a0decc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:17.862331 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-4734a268-9f22-4fac-836b-88a3a0decc01 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:17.863724 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4734a268-9f22-4fac-836b-88a3a0decc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:17.906639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:17.971839 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 679/1362] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:18.275041 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fba67-7eaa-4b1f-af59-75d3174c2dd7 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:45:18.276610 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3f2fba67-7eaa-4b1f-af59-75d3174c2dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:18.473249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fba67-7eaa-4b1f-af59-75d3174c2dd7 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:45:18.484004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:18.594331 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fbed7-616d-46df-b0fd-eedd17f15313 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:45:18.594918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fbed7-616d-46df-b0fd-eedd17f15313 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:18.595198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fbed7-616d-46df-b0fd-eedd17f15313 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:18.595468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fbed7-616d-46df-b0fd-eedd17f15313 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:18.597118 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2d6fbed7-616d-46df-b0fd-eedd17f15313 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:18.698267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6fbed7-616d-46df-b0fd-eedd17f15313 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:45:18.699612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6fbed7-616d-46df-b0fd-eedd17f15313 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:45:18.718425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 684/1363] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:45:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:45:21.238501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec734c-059e-49d7-a162-7fecaddd6767 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:21.239069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec734c-059e-49d7-a162-7fecaddd6767 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:21.239183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec734c-059e-49d7-a162-7fecaddd6767 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:21.240615 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 680/1364] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:21.246108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84c350-da28-4dda-b857-dd58bfc60e23 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:21.246572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84c350-da28-4dda-b857-dd58bfc60e23 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:21.246660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd84c350-da28-4dda-b857-dd58bfc60e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:21.263284 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:21.318734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-dd84c350-da28-4dda-b857-dd58bfc60e23 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:21.320057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-dd84c350-da28-4dda-b857-dd58bfc60e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:21.427053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 685/1365] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:21.721422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147ff74-53a3-4941-8964-0a8cc95fd88f None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:21.721715 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147ff74-53a3-4941-8964-0a8cc95fd88f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:21.721984 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147ff74-53a3-4941-8964-0a8cc95fd88f None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:21.724014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 681/1366] 10.222.0.60 () {54 vars in 965 bytes} [Mon Sep 19 14:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:21.729513 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ad793-6755-41b5-a590-ace4a9fe454a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:21.729870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ad793-6755-41b5-a590-ace4a9fe454a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:21.730217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898ad793-6755-41b5-a590-ace4a9fe454a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:21.808588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-898ad793-6755-41b5-a590-ace4a9fe454a None None] MFA Rules not processed for user `59b5d2fd362b417c9b9be75f88a667ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:21.809786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-898ad793-6755-41b5-a590-ace4a9fe454a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:21.846319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:21.940362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 686/1367] 10.222.0.60 () {58 vars in 1062 bytes} [Mon Sep 19 14:45:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:21.948492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147ff74-53a3-4941-8964-0a8cc95fd88f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:45:21.950416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0147ff74-53a3-4941-8964-0a8cc95fd88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:22.115694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147ff74-53a3-4941-8964-0a8cc95fd88f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:45:22.139054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:22.253070 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82aea7-d735-4bf3-ba6f-e19a05fe2e36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:45:22.253807 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82aea7-d735-4bf3-ba6f-e19a05fe2e36 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:22.254081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82aea7-d735-4bf3-ba6f-e19a05fe2e36 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:22.255141 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82aea7-d735-4bf3-ba6f-e19a05fe2e36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:22.256865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ee82aea7-d735-4bf3-ba6f-e19a05fe2e36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:22.344528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bc288-9a4d-4ad8-9766-39e7cbdec414 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:22.345061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bc288-9a4d-4ad8-9766-39e7cbdec414 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:22.345061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4bc288-9a4d-4ad8-9766-39e7cbdec414 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:22.346437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 687/1368] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:22.352180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7046a0-a446-4518-a278-31bf72617b49 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:22.352736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7046a0-a446-4518-a278-31bf72617b49 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:22.353235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7046a0-a446-4518-a278-31bf72617b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:22.369744 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82aea7-d735-4bf3-ba6f-e19a05fe2e36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:45:22.371360 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82aea7-d735-4bf3-ba6f-e19a05fe2e36 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:45:22.391069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 682/1369] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:45:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:45:22.431873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-fd7046a0-a446-4518-a278-31bf72617b49 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:22.432981 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fd7046a0-a446-4518-a278-31bf72617b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:22.557409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:22.568747 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 688/1370] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:22.878057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4258fe-7cc6-42f1-9e56-bc4b0a16c517 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:22.878347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4258fe-7cc6-42f1-9e56-bc4b0a16c517 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:22.878626 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4258fe-7cc6-42f1-9e56-bc4b0a16c517 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:22.880327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 683/1371] 10.222.0.60 () {54 vars in 962 bytes} [Mon Sep 19 14:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:22.885810 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004ed67-3bce-4462-a90f-a46e13b134ad None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:22.886162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004ed67-3bce-4462-a90f-a46e13b134ad None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:22.886515 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004ed67-3bce-4462-a90f-a46e13b134ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:22.988232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0004ed67-3bce-4462-a90f-a46e13b134ad None None] MFA Rules not processed for user `006fe9ba3b0a40f29fa2be7620e548c4`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:22.990794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0004ed67-3bce-4462-a90f-a46e13b134ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:23.087408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 689/1372] 10.222.0.60 () {58 vars in 1059 bytes} [Mon Sep 19 14:45:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:23.096953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4258fe-7cc6-42f1-9e56-bc4b0a16c517 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:45:23.098518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-db4258fe-7cc6-42f1-9e56-bc4b0a16c517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:23.128778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:23.214954 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4258fe-7cc6-42f1-9e56-bc4b0a16c517 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:45:23.308943 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822f5be-b6ae-489c-b09a-fb3091c7b6f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:45:23.309865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822f5be-b6ae-489c-b09a-fb3091c7b6f7 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:23.310246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822f5be-b6ae-489c-b09a-fb3091c7b6f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:23.310710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822f5be-b6ae-489c-b09a-fb3091c7b6f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:23.313256 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1822f5be-b6ae-489c-b09a-fb3091c7b6f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:23.444648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:23.483810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822f5be-b6ae-489c-b09a-fb3091c7b6f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:45:23.485289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822f5be-b6ae-489c-b09a-fb3091c7b6f7 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:45:23.505758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 684/1373] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:45:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:45:24.637716 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088017b2-1987-4222-9926-3412e8aa737c None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:24.638242 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088017b2-1987-4222-9926-3412e8aa737c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:24.638400 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088017b2-1987-4222-9926-3412e8aa737c None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:24.640726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 690/1374] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:45:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:24.646947 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed3b12-2afc-4f8e-9d9a-620d1e89b03b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:24.647593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed3b12-2afc-4f8e-9d9a-620d1e89b03b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:24.647593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed3b12-2afc-4f8e-9d9a-620d1e89b03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:24.713981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-33ed3b12-2afc-4f8e-9d9a-620d1e89b03b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:24.715205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-33ed3b12-2afc-4f8e-9d9a-620d1e89b03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:24.813720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 685/1375] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:25.109441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088017b2-1987-4222-9926-3412e8aa737c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:45:25.111073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-088017b2-1987-4222-9926-3412e8aa737c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:25.203595 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:25.271302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088017b2-1987-4222-9926-3412e8aa737c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:45:25.394320 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f89759-0bca-4de1-a00b-0f869c048ae5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:45:25.394937 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f89759-0bca-4de1-a00b-0f869c048ae5 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:25.395289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f89759-0bca-4de1-a00b-0f869c048ae5 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:25.395564 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f89759-0bca-4de1-a00b-0f869c048ae5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:25.397191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-33f89759-0bca-4de1-a00b-0f869c048ae5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:25.459615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:25.504707 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f89759-0bca-4de1-a00b-0f869c048ae5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:45:25.506312 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f89759-0bca-4de1-a00b-0f869c048ae5 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:45:25.525794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 691/1376] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:45:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:45:27.777365 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9770df8-a2ae-491e-8f06-306be3f4217e None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:27.777960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9770df8-a2ae-491e-8f06-306be3f4217e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:27.777960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9770df8-a2ae-491e-8f06-306be3f4217e None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:27.779396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 686/1377] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:27.784482 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b2202-792c-41e3-a556-cda0003eb3eb None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:27.784890 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b2202-792c-41e3-a556-cda0003eb3eb None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:27.785085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b2202-792c-41e3-a556-cda0003eb3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:27.860239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3c8b2202-792c-41e3-a556-cda0003eb3eb None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:27.861582 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3c8b2202-792c-41e3-a556-cda0003eb3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:27.971863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 692/1378] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:28.258901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9770df8-a2ae-491e-8f06-306be3f4217e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:45:28.260455 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e9770df8-a2ae-491e-8f06-306be3f4217e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:28.299786 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:28.415467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9770df8-a2ae-491e-8f06-306be3f4217e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:45:28.538705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:28.548023 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345427fe-cd34-4a92-8d19-c146fd5332c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:45:28.548685 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345427fe-cd34-4a92-8d19-c146fd5332c8 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:28.549032 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345427fe-cd34-4a92-8d19-c146fd5332c8 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:28.549376 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345427fe-cd34-4a92-8d19-c146fd5332c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:28.551311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-345427fe-cd34-4a92-8d19-c146fd5332c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:28.650331 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345427fe-cd34-4a92-8d19-c146fd5332c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:45:28.652070 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345427fe-cd34-4a92-8d19-c146fd5332c8 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:45:28.671539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 687/1379] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:45:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:45:30.915026 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ad53d-7b86-4869-9a88-c3c71d0d8022 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:30.915720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ad53d-7b86-4869-9a88-c3c71d0d8022 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:30.916372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ad53d-7b86-4869-9a88-c3c71d0d8022 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:30.919710 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 693/1380] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:45:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:30.924084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f73c0d-6db8-4053-82c5-2f42fa2561eb None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:30.924550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f73c0d-6db8-4053-82c5-2f42fa2561eb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:30.924663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f73c0d-6db8-4053-82c5-2f42fa2561eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:31.020012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-48f73c0d-6db8-4053-82c5-2f42fa2561eb None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:31.021190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-48f73c0d-6db8-4053-82c5-2f42fa2561eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:31.112366 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:31.147561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 688/1381] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:31.465543 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ad53d-7b86-4869-9a88-c3c71d0d8022 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:45:31.467064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ea7ad53d-7b86-4869-9a88-c3c71d0d8022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:31.496717 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:31.626566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ad53d-7b86-4869-9a88-c3c71d0d8022 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:45:31.735457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:31.755363 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125eebe-caa1-4c67-9159-c8beac71a344 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:45:31.756021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125eebe-caa1-4c67-9159-c8beac71a344 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:31.756304 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125eebe-caa1-4c67-9159-c8beac71a344 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:31.756570 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125eebe-caa1-4c67-9159-c8beac71a344 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:31.758196 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6125eebe-caa1-4c67-9159-c8beac71a344 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:31.853097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125eebe-caa1-4c67-9159-c8beac71a344 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:45:31.854699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125eebe-caa1-4c67-9159-c8beac71a344 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:45:31.874057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 694/1382] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:45:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:45:34.130315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413aad3-089f-4745-b0cc-4f0ffbf3a454 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:34.131275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413aad3-089f-4745-b0cc-4f0ffbf3a454 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:34.131562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413aad3-089f-4745-b0cc-4f0ffbf3a454 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:34.133456 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 689/1383] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:45:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:34.139856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131a32d-e969-4adf-abb1-cfd4bf65b032 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:34.140526 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131a32d-e969-4adf-abb1-cfd4bf65b032 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:34.140526 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131a32d-e969-4adf-abb1-cfd4bf65b032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:34.229299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1131a32d-e969-4adf-abb1-cfd4bf65b032 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:34.230857 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1131a32d-e969-4adf-abb1-cfd4bf65b032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:34.303989 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:34.346329 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 695/1384] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:45:34.640428 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413aad3-089f-4745-b0cc-4f0ffbf3a454 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:45:34.642039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6413aad3-089f-4745-b0cc-4f0ffbf3a454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:34.801660 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413aad3-089f-4745-b0cc-4f0ffbf3a454 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:45:34.857526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:34.924072 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98015919-8a29-4deb-a61b-aa456b4bba1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:45:34.924682 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98015919-8a29-4deb-a61b-aa456b4bba1b service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:34.924980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98015919-8a29-4deb-a61b-aa456b4bba1b service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:34.925250 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98015919-8a29-4deb-a61b-aa456b4bba1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:34.926848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-98015919-8a29-4deb-a61b-aa456b4bba1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:35.028966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98015919-8a29-4deb-a61b-aa456b4bba1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:45:35.030657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98015919-8a29-4deb-a61b-aa456b4bba1b service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:45:35.049655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 690/1385] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:45:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:45:37.457506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622c09a-7147-4f50-9f2e-881bf6956696 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:37.458116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622c09a-7147-4f50-9f2e-881bf6956696 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:37.458116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622c09a-7147-4f50-9f2e-881bf6956696 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:37.459403 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 696/1386] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:37.464838 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3093f6-8099-4940-8d87-26fa38c5692d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:37.465303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3093f6-8099-4940-8d87-26fa38c5692d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:37.465504 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3093f6-8099-4940-8d87-26fa38c5692d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:37.538219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5b3093f6-8099-4940-8d87-26fa38c5692d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:37.539430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5b3093f6-8099-4940-8d87-26fa38c5692d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:37.548357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:37.638493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 691/1387] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:37.926693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622c09a-7147-4f50-9f2e-881bf6956696 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:45:37.928271 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9622c09a-7147-4f50-9f2e-881bf6956696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:38.077866 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622c09a-7147-4f50-9f2e-881bf6956696 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:45:38.129665 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:38.198206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13716e82-7956-4ee3-9316-a109c96ba6c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:45:38.198727 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13716e82-7956-4ee3-9316-a109c96ba6c2 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:38.199007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13716e82-7956-4ee3-9316-a109c96ba6c2 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:38.199283 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13716e82-7956-4ee3-9316-a109c96ba6c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:38.200934 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-13716e82-7956-4ee3-9316-a109c96ba6c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:38.295273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13716e82-7956-4ee3-9316-a109c96ba6c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:45:38.297036 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13716e82-7956-4ee3-9316-a109c96ba6c2 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:45:38.317778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 697/1388] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:45:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:45:39.037694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d1f25-d202-40ec-9dc9-0520bf3b6c32 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:39.038275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d1f25-d202-40ec-9dc9-0520bf3b6c32 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:39.038275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d1f25-d202-40ec-9dc9-0520bf3b6c32 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:39.039570 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 692/1389] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:39.044941 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0766c5-ad8c-40bd-8086-fd49b565af89 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:39.045405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0766c5-ad8c-40bd-8086-fd49b565af89 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:39.045534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0766c5-ad8c-40bd-8086-fd49b565af89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:39.118668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:39.121543 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-cb0766c5-ad8c-40bd-8086-fd49b565af89 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:39.122762 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cb0766c5-ad8c-40bd-8086-fd49b565af89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:39.221661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 698/1390] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:45:39.530251 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9d24b-db38-4725-ac69-2c2f0307a771 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:39.530566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9d24b-db38-4725-ac69-2c2f0307a771 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:39.530845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9d24b-db38-4725-ac69-2c2f0307a771 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:39.532402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 693/1391] 10.222.0.60 () {54 vars in 962 bytes} [Mon Sep 19 14:45:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:39.537662 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cdce7-66a4-45e2-a95e-caf07e6e0d08 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:39.538028 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cdce7-66a4-45e2-a95e-caf07e6e0d08 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:39.538400 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cdce7-66a4-45e2-a95e-caf07e6e0d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:39.615442 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-331cdce7-66a4-45e2-a95e-caf07e6e0d08 None None] MFA Rules not processed for user `caab7aaa72634b358d717768fbcb48d8`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:39.620995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-331cdce7-66a4-45e2-a95e-caf07e6e0d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:39.696317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:39.703442 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 699/1392] 10.222.0.60 () {58 vars in 1059 bytes} [Mon Sep 19 14:45:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:39.713520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9d24b-db38-4725-ac69-2c2f0307a771 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:45:39.715077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-10b9d24b-db38-4725-ac69-2c2f0307a771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:39.822353 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9d24b-db38-4725-ac69-2c2f0307a771 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:45:39.860866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:39.900631 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb6d34-d6bf-4c86-8f75-a8af4a41b9de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caab7aaa72634b358d717768fbcb48d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:45:39.901244 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb6d34-d6bf-4c86-8f75-a8af4a41b9de service glance] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:39.901525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb6d34-d6bf-4c86-8f75-a8af4a41b9de service glance] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:39.901795 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb6d34-d6bf-4c86-8f75-a8af4a41b9de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:39.903491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-76bb6d34-d6bf-4c86-8f75-a8af4a41b9de service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:40.017175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb6d34-d6bf-4c86-8f75-a8af4a41b9de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:45:40.019079 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb6d34-d6bf-4c86-8f75-a8af4a41b9de service glance] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:45:40.040752 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 694/1393] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:45:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:45:40.529881 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55665802-ad52-4576-8ba4-d3c610bcf270 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:40.530466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55665802-ad52-4576-8ba4-d3c610bcf270 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:40.530466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55665802-ad52-4576-8ba4-d3c610bcf270 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:40.532130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 700/1394] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:45:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:40.537522 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c842-0367-4351-8d4d-fee3d4e3779b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:40.537808 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c842-0367-4351-8d4d-fee3d4e3779b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:40.538080 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c842-0367-4351-8d4d-fee3d4e3779b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:40.607438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-dc24c842-0367-4351-8d4d-fee3d4e3779b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:40.608735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dc24c842-0367-4351-8d4d-fee3d4e3779b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:40.631725 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:40.719812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 695/1395] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:45:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:40.791620 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793762e-4830-43ac-a706-8d2a9dfc6ea6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:40.791951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793762e-4830-43ac-a706-8d2a9dfc6ea6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:40.792233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793762e-4830-43ac-a706-8d2a9dfc6ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:40.860324 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5793762e-4830-43ac-a706-8d2a9dfc6ea6 None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:40.861555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5793762e-4830-43ac-a706-8d2a9dfc6ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:40.952355 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 701/1396] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Sep 19 14:45:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:40.996407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514aa9f-591d-4170-9b70-80ffb5abd2a5 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:40.996407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514aa9f-591d-4170-9b70-80ffb5abd2a5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:40.996666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1514aa9f-591d-4170-9b70-80ffb5abd2a5 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:40.998188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 696/1397] 10.222.0.60 () {54 vars in 943 bytes} [Mon Sep 19 14:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:41.006209 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ccf55-fa33-4ed8-a848-cc9c2f61868f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:41.006534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ccf55-fa33-4ed8-a848-cc9c2f61868f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:41.006851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ccf55-fa33-4ed8-a848-cc9c2f61868f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:41.009852 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514aa9f-591d-4170-9b70-80ffb5abd2a5 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:45:41.011601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1514aa9f-591d-4170-9b70-80ffb5abd2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:41.086714 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e53ccf55-fa33-4ed8-a848-cc9c2f61868f None None] MFA Rules not processed for user `bfc14d32f4314c6e93d20a9c5a45c220`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:41.088088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e53ccf55-fa33-4ed8-a848-cc9c2f61868f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:41.120816 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:41.164463 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1514aa9f-591d-4170-9b70-80ffb5abd2a5 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:45:41.167695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:41.205199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 702/1398] 10.222.0.60 () {58 vars in 1040 bytes} [Mon Sep 19 14:45:41 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:41.214954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ccf55-fa33-4ed8-a848-cc9c2f61868f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:45:41.225232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ccf55-fa33-4ed8-a848-cc9c2f61868f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:45:41.290536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a7338-3710-4ac5-8b02-3f8efad44cd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:45:41.291136 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a7338-3710-4ac5-8b02-3f8efad44cd7 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:41.291416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a7338-3710-4ac5-8b02-3f8efad44cd7 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:41.291682 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a7338-3710-4ac5-8b02-3f8efad44cd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:41.293343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-845a7338-3710-4ac5-8b02-3f8efad44cd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:41.341730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8175f7-3085-40b6-bb36-52bb70f5ec25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:45:41.342343 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8175f7-3085-40b6-bb36-52bb70f5ec25 service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:41.342623 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8175f7-3085-40b6-bb36-52bb70f5ec25 service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:41.342896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8175f7-3085-40b6-bb36-52bb70f5ec25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:41.344604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4d8175f7-3085-40b6-bb36-52bb70f5ec25 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:41.378691 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:41.388234 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a7338-3710-4ac5-8b02-3f8efad44cd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:45:41.389847 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a7338-3710-4ac5-8b02-3f8efad44cd7 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:45:41.407691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 697/1399] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:45:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:45:41.433492 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8175f7-3085-40b6-bb36-52bb70f5ec25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:45:41.435303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8175f7-3085-40b6-bb36-52bb70f5ec25 service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:45:41.443859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 703/1400] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 14:45:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 14:45:43.699362 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabac0a-1040-48f0-9148-937e529d4e8d None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:43.700192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabac0a-1040-48f0-9148-937e529d4e8d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:43.700192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabac0a-1040-48f0-9148-937e529d4e8d None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:43.701603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 698/1401] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:45:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:43.707875 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f2399-fbc2-477a-8c67-4a48bc37cc7f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:43.709359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f2399-fbc2-477a-8c67-4a48bc37cc7f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:43.709359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f2399-fbc2-477a-8c67-4a48bc37cc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:43.784129 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c75f2399-fbc2-477a-8c67-4a48bc37cc7f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:43.786586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c75f2399-fbc2-477a-8c67-4a48bc37cc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:43.920087 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 704/1402] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:44.215193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabac0a-1040-48f0-9148-937e529d4e8d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:45:44.217067 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1aabac0a-1040-48f0-9148-937e529d4e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:44.227426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:44.400497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabac0a-1040-48f0-9148-937e529d4e8d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:45:44.501818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:44.546106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f444a2-51cd-4189-9c8b-59a34c57882b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:45:44.546827 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f444a2-51cd-4189-9c8b-59a34c57882b service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:44.547162 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f444a2-51cd-4189-9c8b-59a34c57882b service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:44.547514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f444a2-51cd-4189-9c8b-59a34c57882b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:44.549291 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-46f444a2-51cd-4189-9c8b-59a34c57882b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:44.676751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f444a2-51cd-4189-9c8b-59a34c57882b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:45:44.678835 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f444a2-51cd-4189-9c8b-59a34c57882b service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:45:44.702733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 699/1403] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:45:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:45:46.942656 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92c880-fca1-4040-8f46-75cc8349faa2 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:46.943243 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92c880-fca1-4040-8f46-75cc8349faa2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:46.943243 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92c880-fca1-4040-8f46-75cc8349faa2 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:46.944556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 705/1404] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:46.950107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfd644-2a99-4ef9-ba08-81594168a797 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:46.950610 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfd644-2a99-4ef9-ba08-81594168a797 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:46.950728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbfd644-2a99-4ef9-ba08-81594168a797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:47.025408 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0cbfd644-2a99-4ef9-ba08-81594168a797 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:47.026611 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0cbfd644-2a99-4ef9-ba08-81594168a797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:47.079113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:47.147219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 700/1405] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:47.499326 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92c880-fca1-4040-8f46-75cc8349faa2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:45:47.499724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-af92c880-fca1-4040-8f46-75cc8349faa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:47.530109 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:47.667021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92c880-fca1-4040-8f46-75cc8349faa2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:45:47.778468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:47.799195 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b72181-b1b3-48fe-8aac-c7431377a0e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:45:47.799818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b72181-b1b3-48fe-8aac-c7431377a0e3 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:47.800153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b72181-b1b3-48fe-8aac-c7431377a0e3 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:47.800414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b72181-b1b3-48fe-8aac-c7431377a0e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:47.802111 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b6b72181-b1b3-48fe-8aac-c7431377a0e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:47.899205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b72181-b1b3-48fe-8aac-c7431377a0e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:45:47.900826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b72181-b1b3-48fe-8aac-c7431377a0e3 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:45:47.923464 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 706/1406] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:45:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:45:50.178068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b42a00-e93e-4efa-8d6d-c61e82001139 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:50.178905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b42a00-e93e-4efa-8d6d-c61e82001139 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:50.178905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b42a00-e93e-4efa-8d6d-c61e82001139 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:50.180653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 701/1407] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:45:50.186059 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959781d4-381a-407b-aea6-9deb1503d020 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:50.186485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959781d4-381a-407b-aea6-9deb1503d020 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:50.186593 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959781d4-381a-407b-aea6-9deb1503d020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:50.260957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-959781d4-381a-407b-aea6-9deb1503d020 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:45:50.262500 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-959781d4-381a-407b-aea6-9deb1503d020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:50.330444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:50.374133 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 707/1408] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:45:50.673903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b42a00-e93e-4efa-8d6d-c61e82001139 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:45:50.675849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-99b42a00-e93e-4efa-8d6d-c61e82001139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:50.840084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b42a00-e93e-4efa-8d6d-c61e82001139 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:45:50.873300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:45:50.968628 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdc4bb-3542-449c-bd89-fcaad76e14ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:45:50.969155 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdc4bb-3542-449c-bd89-fcaad76e14ca service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:45:50.969435 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdc4bb-3542-449c-bd89-fcaad76e14ca service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:45:50.969707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdc4bb-3542-449c-bd89-fcaad76e14ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:45:50.971300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4bcdc4bb-3542-449c-bd89-fcaad76e14ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:45:51.073717 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdc4bb-3542-449c-bd89-fcaad76e14ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:45:51.073717 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdc4bb-3542-449c-bd89-fcaad76e14ca service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:45:51.095105 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 702/1409] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:00.372583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=16 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:01.090357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=86 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:05.002944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4374e0-0118-4acb-b780-3983d7a44a50 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:05.003622 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4374e0-0118-4acb-b780-3983d7a44a50 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:05.003763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4374e0-0118-4acb-b780-3983d7a44a50 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:05.006104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 708/1410] 10.222.0.60 () {54 vars in 918 bytes} [Mon Sep 19 14:46:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:05.011253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948fad1-48db-4ed3-b9c4-625bcd754d22 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:05.011671 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948fad1-48db-4ed3-b9c4-625bcd754d22 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:05.011875 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0948fad1-48db-4ed3-b9c4-625bcd754d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:05.092048 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0948fad1-48db-4ed3-b9c4-625bcd754d22 None None] MFA Rules not processed for user `568189358b154b66a729fae2f0411d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:05.093290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0948fad1-48db-4ed3-b9c4-625bcd754d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:05.199110 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 703/1411] 10.222.0.60 () {58 vars in 1015 bytes} [Mon Sep 19 14:46:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:05.214352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512c28d-7fe7-4959-a193-1d593f5043b2 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:05.214640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512c28d-7fe7-4959-a193-1d593f5043b2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:05.214918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9512c28d-7fe7-4959-a193-1d593f5043b2 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:05.216267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 709/1412] 10.222.0.60 () {54 vars in 918 bytes} [Mon Sep 19 14:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:05.221517 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ed4c8-b0a0-41e2-a515-18575839a772 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:05.221805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ed4c8-b0a0-41e2-a515-18575839a772 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:05.222586 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ed4c8-b0a0-41e2-a515-18575839a772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:05.295235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:05.298427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9c0ed4c8-b0a0-41e2-a515-18575839a772 None None] MFA Rules not processed for user `59b5d2fd362b417c9b9be75f88a667ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:05.300768 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9c0ed4c8-b0a0-41e2-a515-18575839a772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:05.429972 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 704/1413] 10.222.0.60 () {58 vars in 1015 bytes} [Mon Sep 19 14:46:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:08.034979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2527d37-b56f-49e4-b2df-d741698f2533 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:08.035566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2527d37-b56f-49e4-b2df-d741698f2533 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:08.035566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2527d37-b56f-49e4-b2df-d741698f2533 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:08.036888 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 14:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:08.042145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d99b7-069c-4cb9-bd4f-6ff7dfb67506 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:08.042593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d99b7-069c-4cb9-bd4f-6ff7dfb67506 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:08.042700 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d99b7-069c-4cb9-bd4f-6ff7dfb67506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:08.112342 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a11d99b7-069c-4cb9-bd4f-6ff7dfb67506 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:08.113742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a11d99b7-069c-4cb9-bd4f-6ff7dfb67506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:08.168387 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:08.212731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 14:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:08.502916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2527d37-b56f-49e4-b2df-d741698f2533 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:08.504497 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b2527d37-b56f-49e4-b2df-d741698f2533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:08.650456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2527d37-b56f-49e4-b2df-d741698f2533 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:08.726638 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:08.767940 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3ff9a-c0f6-4763-b460-1b4f2ae92e5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:08.768477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3ff9a-c0f6-4763-b460-1b4f2ae92e5a service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:08.768762 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3ff9a-c0f6-4763-b460-1b4f2ae92e5a service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:08.769715 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3ff9a-c0f6-4763-b460-1b4f2ae92e5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:08.771357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-08f3ff9a-c0f6-4763-b460-1b4f2ae92e5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:08.862435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3ff9a-c0f6-4763-b460-1b4f2ae92e5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:08.864183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3ff9a-c0f6-4763-b460-1b4f2ae92e5a service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:08.882777 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 711/1416] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:10.036056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f709d54-5766-42b9-9707-164ee6e04056 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:10.036621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f709d54-5766-42b9-9707-164ee6e04056 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:10.036621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f709d54-5766-42b9-9707-164ee6e04056 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:10.037991 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 14:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:10.044439 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85449560-f43c-4cd4-9008-8bbe96363f0e None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:10.044873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85449560-f43c-4cd4-9008-8bbe96363f0e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:10.044976 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85449560-f43c-4cd4-9008-8bbe96363f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:10.127332 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-85449560-f43c-4cd4-9008-8bbe96363f0e None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:10.129250 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-85449560-f43c-4cd4-9008-8bbe96363f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:10.156749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:10.228691 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 14:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:10.377814 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f709d54-5766-42b9-9707-164ee6e04056 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:10.379448 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2f709d54-5766-42b9-9707-164ee6e04056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:10.524693 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f709d54-5766-42b9-9707-164ee6e04056 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:10.565863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:10.639350 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953097d-2047-4f9d-905e-d04c0ad1052e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:10.640102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953097d-2047-4f9d-905e-d04c0ad1052e service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:10.640289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953097d-2047-4f9d-905e-d04c0ad1052e service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:10.640542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953097d-2047-4f9d-905e-d04c0ad1052e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:10.642149 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c953097d-2047-4f9d-905e-d04c0ad1052e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:10.738175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953097d-2047-4f9d-905e-d04c0ad1052e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:10.739786 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953097d-2047-4f9d-905e-d04c0ad1052e service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:10.758757 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 707/1419] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:11.884422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2fbdc-ec14-4534-8458-5440718496b0 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:11.885059 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2fbdc-ec14-4534-8458-5440718496b0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:11.885059 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2fbdc-ec14-4534-8458-5440718496b0 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:11.886320 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 14:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:11.892992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de35e11-b4a3-4e1e-9741-e465fac33da0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:11.893528 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de35e11-b4a3-4e1e-9741-e465fac33da0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:11.893528 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de35e11-b4a3-4e1e-9741-e465fac33da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:11.947845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:11.970765 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3de35e11-b4a3-4e1e-9741-e465fac33da0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:11.972022 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3de35e11-b4a3-4e1e-9741-e465fac33da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:12.071260 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 14:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:12.221204 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2fbdc-ec14-4534-8458-5440718496b0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:12.222716 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4ac2fbdc-ec14-4534-8458-5440718496b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:12.370472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2fbdc-ec14-4534-8458-5440718496b0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:12.384648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:12.494106 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee98a1-9304-4a7b-a378-a7ad4146f0db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:12.494628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee98a1-9304-4a7b-a378-a7ad4146f0db service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:12.494906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee98a1-9304-4a7b-a378-a7ad4146f0db service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:12.495178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee98a1-9304-4a7b-a378-a7ad4146f0db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:12.496810 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-39ee98a1-9304-4a7b-a378-a7ad4146f0db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:12.594728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee98a1-9304-4a7b-a378-a7ad4146f0db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:12.596677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee98a1-9304-4a7b-a378-a7ad4146f0db service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:12.615532 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 714/1422] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:46:13.730250 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e1da2-0a82-4be1-bedc-2f3eef8aade3 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:13.730859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e1da2-0a82-4be1-bedc-2f3eef8aade3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:13.730859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26e1da2-0a82-4be1-bedc-2f3eef8aade3 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:13.732153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 14:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:13.738971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f2c06-0dad-4807-be10-f4df3aabff47 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:13.739507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f2c06-0dad-4807-be10-f4df3aabff47 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:13.739507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f2c06-0dad-4807-be10-f4df3aabff47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:13.760281 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:13.816820 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-7d8f2c06-0dad-4807-be10-f4df3aabff47 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:13.818105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7d8f2c06-0dad-4807-be10-f4df3aabff47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:13.913632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 14:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:14.063526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e1da2-0a82-4be1-bedc-2f3eef8aade3 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:14.064994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d26e1da2-0a82-4be1-bedc-2f3eef8aade3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:14.175073 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:14.211721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26e1da2-0a82-4be1-bedc-2f3eef8aade3 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:14.326662 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc681d-8cd5-4f5e-b0c5-5e08ba309e28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:14.327191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc681d-8cd5-4f5e-b0c5-5e08ba309e28 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:14.327466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc681d-8cd5-4f5e-b0c5-5e08ba309e28 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:14.327736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc681d-8cd5-4f5e-b0c5-5e08ba309e28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:14.329356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cfcc681d-8cd5-4f5e-b0c5-5e08ba309e28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:14.417747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:14.432172 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc681d-8cd5-4f5e-b0c5-5e08ba309e28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:14.433777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc681d-8cd5-4f5e-b0c5-5e08ba309e28 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:14.452006 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 710/1425] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:15.617844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baeb1eb-2f6e-4992-bbe4-7dc2c456e215 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:15.618447 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baeb1eb-2f6e-4992-bbe4-7dc2c456e215 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:15.618447 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baeb1eb-2f6e-4992-bbe4-7dc2c456e215 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:15.619730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 14:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:15.626366 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f8419-80db-4a6c-8901-09e668fc5b76 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:15.626858 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f8419-80db-4a6c-8901-09e668fc5b76 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:15.626971 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f8419-80db-4a6c-8901-09e668fc5b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:15.704141 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a43f8419-80db-4a6c-8901-09e668fc5b76 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:15.705377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a43f8419-80db-4a6c-8901-09e668fc5b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:15.803557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 14:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:15.953344 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baeb1eb-2f6e-4992-bbe4-7dc2c456e215 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:15.954686 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9baeb1eb-2f6e-4992-bbe4-7dc2c456e215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:16.040361 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:16.104129 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baeb1eb-2f6e-4992-bbe4-7dc2c456e215 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:16.224194 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5e83d-d420-480b-9d59-5568bf1a5e82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:16.224808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5e83d-d420-480b-9d59-5568bf1a5e82 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:16.225089 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5e83d-d420-480b-9d59-5568bf1a5e82 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:16.225357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5e83d-d420-480b-9d59-5568bf1a5e82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:16.227011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-13e5e83d-d420-480b-9d59-5568bf1a5e82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:16.275856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:16.318640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5e83d-d420-480b-9d59-5568bf1a5e82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:16.320270 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5e83d-d420-480b-9d59-5568bf1a5e82 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:16.338820 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 717/1428] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:17.388426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ace108-d017-4728-98cc-43fc900d4cc4 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:17.388958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ace108-d017-4728-98cc-43fc900d4cc4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:17.388958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ace108-d017-4728-98cc-43fc900d4cc4 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:17.390199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 14:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:17.397063 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab072ef-c612-471a-8f0a-04a57a8079a7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:17.397526 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab072ef-c612-471a-8f0a-04a57a8079a7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:17.397640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab072ef-c612-471a-8f0a-04a57a8079a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:17.475359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0ab072ef-c612-471a-8f0a-04a57a8079a7 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:17.476611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0ab072ef-c612-471a-8f0a-04a57a8079a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:17.572214 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 14:46:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:17.725257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ace108-d017-4728-98cc-43fc900d4cc4 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:17.726771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d8ace108-d017-4728-98cc-43fc900d4cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:17.777929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:17.878568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ace108-d017-4728-98cc-43fc900d4cc4 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:17.992231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74e8b3-8ca5-4c2b-b9ae-3d39161e3756 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:17.992771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74e8b3-8ca5-4c2b-b9ae-3d39161e3756 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:17.993054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74e8b3-8ca5-4c2b-b9ae-3d39161e3756 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:17.993324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74e8b3-8ca5-4c2b-b9ae-3d39161e3756 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:17.994920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4b74e8b3-8ca5-4c2b-b9ae-3d39161e3756 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:18.002053 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:18.089174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74e8b3-8ca5-4c2b-b9ae-3d39161e3756 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:18.090790 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74e8b3-8ca5-4c2b-b9ae-3d39161e3756 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:18.108580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 713/1431] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:19.174057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3611776-5ced-40d6-abb7-45a0314546b4 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:19.174627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3611776-5ced-40d6-abb7-45a0314546b4 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:19.174627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3611776-5ced-40d6-abb7-45a0314546b4 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:19.176476 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 14:46:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:19.183083 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9514c18-dd1b-40be-8905-8c7c05025665 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:19.183532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9514c18-dd1b-40be-8905-8c7c05025665 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:19.183634 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9514c18-dd1b-40be-8905-8c7c05025665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:19.259201 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b9514c18-dd1b-40be-8905-8c7c05025665 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:19.260478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b9514c18-dd1b-40be-8905-8c7c05025665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:19.330512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:19.363536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 14:46:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:19.514486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3611776-5ced-40d6-abb7-45a0314546b4 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:19.516045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c3611776-5ced-40d6-abb7-45a0314546b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:19.534101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:19.661783 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3611776-5ced-40d6-abb7-45a0314546b4 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:19.749843 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:19.777063 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9039b-eab2-4514-a287-91f810178a92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:19.777672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9039b-eab2-4514-a287-91f810178a92 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:19.777953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9039b-eab2-4514-a287-91f810178a92 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:19.778229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9039b-eab2-4514-a287-91f810178a92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:19.779856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3ad9039b-eab2-4514-a287-91f810178a92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:19.869960 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9039b-eab2-4514-a287-91f810178a92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:19.871614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9039b-eab2-4514-a287-91f810178a92 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:19.894299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 720/1434] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:20.956309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b299df-9f0b-4be3-ba32-e28f20091e18 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:20.956926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b299df-9f0b-4be3-ba32-e28f20091e18 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:20.956926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b299df-9f0b-4be3-ba32-e28f20091e18 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:20.958205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 14:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:20.965307 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2be0d-7569-4a44-8316-887186f9be26 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:20.965307 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2be0d-7569-4a44-8316-887186f9be26 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:20.965307 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2be0d-7569-4a44-8316-887186f9be26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:21.040167 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-70f2be0d-7569-4a44-8316-887186f9be26 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:21.041361 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-70f2be0d-7569-4a44-8316-887186f9be26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:21.083287 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:21.138672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 14:46:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:21.287263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b299df-9f0b-4be3-ba32-e28f20091e18 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:21.288861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-17b299df-9f0b-4be3-ba32-e28f20091e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:21.437361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b299df-9f0b-4be3-ba32-e28f20091e18 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:21.494915 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:21.554470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205ed3d-12a6-4d61-a844-9ee4943d891a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:21.555002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205ed3d-12a6-4d61-a844-9ee4943d891a service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:21.555301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205ed3d-12a6-4d61-a844-9ee4943d891a service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:21.555557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205ed3d-12a6-4d61-a844-9ee4943d891a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:21.557184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e205ed3d-12a6-4d61-a844-9ee4943d891a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:21.648840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205ed3d-12a6-4d61-a844-9ee4943d891a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:21.650463 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205ed3d-12a6-4d61-a844-9ee4943d891a service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:21.668556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 716/1437] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:22.734060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f042b-6b54-4236-92bf-acbc82bb5beb None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:22.734643 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f042b-6b54-4236-92bf-acbc82bb5beb None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:22.734643 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f042b-6b54-4236-92bf-acbc82bb5beb None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:22.736005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 14:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:22.742338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a6414-b57b-436b-9810-a8021536a6e5 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:22.742820 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a6414-b57b-436b-9810-a8021536a6e5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:22.742924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6a6414-b57b-436b-9810-a8021536a6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:22.821704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8c6a6414-b57b-436b-9810-a8021536a6e5 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:22.822914 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8c6a6414-b57b-436b-9810-a8021536a6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:22.824981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:22.924032 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 14:46:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:23.074110 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f042b-6b54-4236-92bf-acbc82bb5beb None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:23.075554 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0b2f042b-6b54-4236-92bf-acbc82bb5beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:23.228646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f042b-6b54-4236-92bf-acbc82bb5beb None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:23.258348 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:23.347562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba3381-d15a-4d0f-a71f-9696bdeab578 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:23.348114 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba3381-d15a-4d0f-a71f-9696bdeab578 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:23.348393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba3381-d15a-4d0f-a71f-9696bdeab578 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:23.348661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba3381-d15a-4d0f-a71f-9696bdeab578 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:23.350237 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f6ba3381-d15a-4d0f-a71f-9696bdeab578 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:23.442271 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba3381-d15a-4d0f-a71f-9696bdeab578 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:23.443978 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba3381-d15a-4d0f-a71f-9696bdeab578 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:23.462308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 723/1440] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:25.521365 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5324f-f697-4fca-82ab-3c6be32ee76b None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:25.521939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5324f-f697-4fca-82ab-3c6be32ee76b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:25.521939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5324f-f697-4fca-82ab-3c6be32ee76b None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:25.523246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 14:46:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:25.528632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810e8fc-5dad-4ecd-a149-5b735e9f1465 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:25.529079 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810e8fc-5dad-4ecd-a149-5b735e9f1465 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:25.529223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f810e8fc-5dad-4ecd-a149-5b735e9f1465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:25.571001 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:25.596334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f810e8fc-5dad-4ecd-a149-5b735e9f1465 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:25.598087 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f810e8fc-5dad-4ecd-a149-5b735e9f1465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:25.695274 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 14:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:25.977106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5324f-f697-4fca-82ab-3c6be32ee76b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:25.978655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4dc5324f-f697-4fca-82ab-3c6be32ee76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:26.084086 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5324f-f697-4fca-82ab-3c6be32ee76b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:26.116050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:26.158970 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f45878-b97d-4a76-98ac-60d0bcd39ee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:26.159574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f45878-b97d-4a76-98ac-60d0bcd39ee8 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:26.159929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f45878-b97d-4a76-98ac-60d0bcd39ee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:26.160154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f45878-b97d-4a76-98ac-60d0bcd39ee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:26.161743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-65f45878-b97d-4a76-98ac-60d0bcd39ee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:26.271056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f45878-b97d-4a76-98ac-60d0bcd39ee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:26.272522 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f45878-b97d-4a76-98ac-60d0bcd39ee8 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:26.290370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 719/1443] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:28.592553 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1d54-9aaf-4861-9307-aaacbadf6984 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:28.593129 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1d54-9aaf-4861-9307-aaacbadf6984 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:28.593129 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1d54-9aaf-4861-9307-aaacbadf6984 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:28.594475 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 14:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:28.599778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4022748-ecea-4944-a719-a8596f350716 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:28.600248 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4022748-ecea-4944-a719-a8596f350716 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:28.600368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4022748-ecea-4944-a719-a8596f350716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:28.667247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b4022748-ecea-4944-a719-a8596f350716 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:28.668520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b4022748-ecea-4944-a719-a8596f350716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:28.681941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:28.764032 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 14:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:29.050035 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a1d54-9aaf-4861-9307-aaacbadf6984 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:29.051579 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-dc9a1d54-9aaf-4861-9307-aaacbadf6984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:29.161848 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a1d54-9aaf-4861-9307-aaacbadf6984 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:29.174842 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:29.240727 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890da373-dd7e-41b5-ad7f-36da29ab5f12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:29.241353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890da373-dd7e-41b5-ad7f-36da29ab5f12 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:29.241675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890da373-dd7e-41b5-ad7f-36da29ab5f12 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:29.242129 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890da373-dd7e-41b5-ad7f-36da29ab5f12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:29.244683 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-890da373-dd7e-41b5-ad7f-36da29ab5f12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:29.363720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890da373-dd7e-41b5-ad7f-36da29ab5f12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:29.365113 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890da373-dd7e-41b5-ad7f-36da29ab5f12 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:29.383853 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 726/1446] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:31.933738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867f083-4083-4478-b4b6-ae043997aaf7 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:31.934322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867f083-4083-4478-b4b6-ae043997aaf7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:31.934322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867f083-4083-4478-b4b6-ae043997aaf7 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:31.935623 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 14:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:31.941084 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbb611-7aab-4983-8efc-8456e16335de None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:31.941521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbb611-7aab-4983-8efc-8456e16335de None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:31.941638 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bbb611-7aab-4983-8efc-8456e16335de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:31.991073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:32.008840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f4bbb611-7aab-4983-8efc-8456e16335de None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:32.010051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f4bbb611-7aab-4983-8efc-8456e16335de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:32.106697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 14:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:46:32.395579 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867f083-4083-4478-b4b6-ae043997aaf7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:32.397222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8867f083-4083-4478-b4b6-ae043997aaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:32.499315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867f083-4083-4478-b4b6-ae043997aaf7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:32.539131 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:32.573447 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01652394-0194-42a8-a9c0-a05c7de48b19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:32.574049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01652394-0194-42a8-a9c0-a05c7de48b19 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:32.574329 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01652394-0194-42a8-a9c0-a05c7de48b19 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:32.574600 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01652394-0194-42a8-a9c0-a05c7de48b19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:32.576271 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-01652394-0194-42a8-a9c0-a05c7de48b19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:32.687736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01652394-0194-42a8-a9c0-a05c7de48b19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:32.689212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01652394-0194-42a8-a9c0-a05c7de48b19 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:32.707349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 722/1449] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:35.014490 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b23d87-7a17-44b8-b700-4280d7646e00 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:35.015085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b23d87-7a17-44b8-b700-4280d7646e00 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:35.015085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b23d87-7a17-44b8-b700-4280d7646e00 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:35.016409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 14:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:35.021723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae852e1c-8683-42cd-93ed-901904dddec1 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:35.022114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae852e1c-8683-42cd-93ed-901904dddec1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:35.022283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae852e1c-8683-42cd-93ed-901904dddec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:35.204837 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ae852e1c-8683-42cd-93ed-901904dddec1 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:35.206066 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ae852e1c-8683-42cd-93ed-901904dddec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:35.226604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:35.301598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 14:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:35.589997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b23d87-7a17-44b8-b700-4280d7646e00 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:35.591530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-20b23d87-7a17-44b8-b700-4280d7646e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:35.695853 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b23d87-7a17-44b8-b700-4280d7646e00 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:35.716524 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:35.770097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e5a5f-7f0e-475f-8470-d1f6f8748466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:35.770711 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e5a5f-7f0e-475f-8470-d1f6f8748466 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:35.770994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e5a5f-7f0e-475f-8470-d1f6f8748466 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:35.771298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e5a5f-7f0e-475f-8470-d1f6f8748466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:35.772945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d34e5a5f-7f0e-475f-8470-d1f6f8748466 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:35.883640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e5a5f-7f0e-475f-8470-d1f6f8748466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:35.885051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e5a5f-7f0e-475f-8470-d1f6f8748466 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:35.903708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 729/1452] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:38.186967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6874bc-e707-4349-a68f-3402942f8c88 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:38.187536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6874bc-e707-4349-a68f-3402942f8c88 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:38.187536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6874bc-e707-4349-a68f-3402942f8c88 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:38.188862 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 14:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:38.194168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ac437-5a32-4f67-88e7-8817586894c7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:38.194602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ac437-5a32-4f67-88e7-8817586894c7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:38.194750 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ac437-5a32-4f67-88e7-8817586894c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:38.258677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:38.261378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1f4ac437-5a32-4f67-88e7-8817586894c7 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:38.263225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1f4ac437-5a32-4f67-88e7-8817586894c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:38.360278 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 14:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:38.649056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6874bc-e707-4349-a68f-3402942f8c88 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:38.650680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1a6874bc-e707-4349-a68f-3402942f8c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:38.753045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6874bc-e707-4349-a68f-3402942f8c88 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:38.801108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:38.825859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9e446f-8131-4f7b-9d25-f63744f95f85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:38.826458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e446f-8131-4f7b-9d25-f63744f95f85 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:38.826733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e446f-8131-4f7b-9d25-f63744f95f85 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:38.827000 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9e446f-8131-4f7b-9d25-f63744f95f85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:38.828604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3b9e446f-8131-4f7b-9d25-f63744f95f85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:38.938812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e446f-8131-4f7b-9d25-f63744f95f85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:38.940810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9e446f-8131-4f7b-9d25-f63744f95f85 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:38.959297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 725/1455] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:41.194294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e39a3-fa75-43b1-a3d8-41e71d0deb4c None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:41.195173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e39a3-fa75-43b1-a3d8-41e71d0deb4c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:41.195173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003e39a3-fa75-43b1-a3d8-41e71d0deb4c None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:41.196735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 14:46:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:41.202079 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac71e6-0f40-4861-ac33-aded1d65e826 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:41.202486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac71e6-0f40-4861-ac33-aded1d65e826 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:41.202633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac71e6-0f40-4861-ac33-aded1d65e826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:41.270572 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-bfac71e6-0f40-4861-ac33-aded1d65e826 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:41.271799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bfac71e6-0f40-4861-ac33-aded1d65e826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:41.301452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:41.370371 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 14:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:41.655964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e39a3-fa75-43b1-a3d8-41e71d0deb4c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:41.657535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-003e39a3-fa75-43b1-a3d8-41e71d0deb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:41.762114 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003e39a3-fa75-43b1-a3d8-41e71d0deb4c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:41.790868 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:41.836762 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89bcb49-18a6-4117-9537-2fc72e9f04c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:41.837372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bcb49-18a6-4117-9537-2fc72e9f04c4 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:41.837654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bcb49-18a6-4117-9537-2fc72e9f04c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:41.837939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89bcb49-18a6-4117-9537-2fc72e9f04c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:41.839526 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f89bcb49-18a6-4117-9537-2fc72e9f04c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:41.950273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89bcb49-18a6-4117-9537-2fc72e9f04c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:41.951618 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89bcb49-18a6-4117-9537-2fc72e9f04c4 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:41.970161 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 732/1458] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:44.198356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a4224-9625-4258-82cf-15b0f6b9c691 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:44.198958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a4224-9625-4258-82cf-15b0f6b9c691 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:44.198958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a4224-9625-4258-82cf-15b0f6b9c691 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:44.200277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 14:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:44.206012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc24e3d-5013-4a0b-83a1-70c59c160514 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:44.206465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc24e3d-5013-4a0b-83a1-70c59c160514 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:44.206568 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc24e3d-5013-4a0b-83a1-70c59c160514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:44.272846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-bfc24e3d-5013-4a0b-83a1-70c59c160514 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:44.273967 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bfc24e3d-5013-4a0b-83a1-70c59c160514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:44.282190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:44.378085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 14:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:44.666640 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a4224-9625-4258-82cf-15b0f6b9c691 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:44.668202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-298a4224-9625-4258-82cf-15b0f6b9c691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:44.784236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a4224-9625-4258-82cf-15b0f6b9c691 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:44.840766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:44.858933 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb6833-eae6-40eb-b3a1-dfeb9742d5b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:44.859543 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb6833-eae6-40eb-b3a1-dfeb9742d5b8 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:44.859841 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb6833-eae6-40eb-b3a1-dfeb9742d5b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:44.860124 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb6833-eae6-40eb-b3a1-dfeb9742d5b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:44.862309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c0bb6833-eae6-40eb-b3a1-dfeb9742d5b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:44.988427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb6833-eae6-40eb-b3a1-dfeb9742d5b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:44.989773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb6833-eae6-40eb-b3a1-dfeb9742d5b8 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:45.007377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 728/1461] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:47.254905 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2d5a7-4ed2-4f9e-8d7a-91b9e76ec5bc None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:47.255477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2d5a7-4ed2-4f9e-8d7a-91b9e76ec5bc None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:47.255477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2d5a7-4ed2-4f9e-8d7a-91b9e76ec5bc None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:47.256825 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 14:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:47.262455 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9a79f-0d22-40c9-8776-f84823773263 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:47.262898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9a79f-0d22-40c9-8776-f84823773263 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:47.263051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe9a79f-0d22-40c9-8776-f84823773263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:47.328492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-cbe9a79f-0d22-40c9-8776-f84823773263 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:47.329711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cbe9a79f-0d22-40c9-8776-f84823773263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:47.368954 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:47.426916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 14:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:47.716699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2d5a7-4ed2-4f9e-8d7a-91b9e76ec5bc None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:47.718267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-17d2d5a7-4ed2-4f9e-8d7a-91b9e76ec5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:47.829087 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2d5a7-4ed2-4f9e-8d7a-91b9e76ec5bc None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:47.867974 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:47.904326 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979699a-3551-4297-931e-4a2768b7b797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:47.904939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979699a-3551-4297-931e-4a2768b7b797 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:47.905225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979699a-3551-4297-931e-4a2768b7b797 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:47.905494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979699a-3551-4297-931e-4a2768b7b797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:47.907095 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d979699a-3551-4297-931e-4a2768b7b797 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:48.016679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979699a-3551-4297-931e-4a2768b7b797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:48.018122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979699a-3551-4297-931e-4a2768b7b797 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:48.036963 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 735/1464] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:50.190702 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4560f5-44f6-4908-a815-1e587f9edd05 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:50.191278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4560f5-44f6-4908-a815-1e587f9edd05 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:50.191278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4560f5-44f6-4908-a815-1e587f9edd05 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:50.192626 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 14:46:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:50.197846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961fa33-0fd5-4774-8536-1ad61fdf991c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:50.198294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961fa33-0fd5-4774-8536-1ad61fdf991c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:50.198394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961fa33-0fd5-4774-8536-1ad61fdf991c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:50.263793 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3961fa33-0fd5-4774-8536-1ad61fdf991c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:50.264977 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3961fa33-0fd5-4774-8536-1ad61fdf991c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:50.282797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:50.363070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 14:46:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:46:50.646349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4560f5-44f6-4908-a815-1e587f9edd05 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:50.647962 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6e4560f5-44f6-4908-a815-1e587f9edd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:50.749872 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4560f5-44f6-4908-a815-1e587f9edd05 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:50.817017 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:50.823816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840ea4f-e1d5-440e-ac7d-70f235f75ffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:50.824454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840ea4f-e1d5-440e-ac7d-70f235f75ffd service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:50.824734 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840ea4f-e1d5-440e-ac7d-70f235f75ffd service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:50.825005 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840ea4f-e1d5-440e-ac7d-70f235f75ffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:50.826688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a840ea4f-e1d5-440e-ac7d-70f235f75ffd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:50.935318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840ea4f-e1d5-440e-ac7d-70f235f75ffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:50.936793 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840ea4f-e1d5-440e-ac7d-70f235f75ffd service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:50.954759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 731/1467] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:46:53.155896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa009f-f179-493b-8f45-db2adbd6087f None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:53.156490 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa009f-f179-493b-8f45-db2adbd6087f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:53.156490 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa009f-f179-493b-8f45-db2adbd6087f None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:53.157807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 14:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:46:53.163909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec893ec-06c0-4216-a26a-d41855f665e2 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:53.164358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec893ec-06c0-4216-a26a-d41855f665e2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:53.164527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec893ec-06c0-4216-a26a-d41855f665e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:53.235212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0ec893ec-06c0-4216-a26a-d41855f665e2 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:46:53.237003 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0ec893ec-06c0-4216-a26a-d41855f665e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:53.283970 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:53.335132 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 14:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:46:53.619404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa009f-f179-493b-8f45-db2adbd6087f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:46:53.620961 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f6aa009f-f179-493b-8f45-db2adbd6087f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:53.734980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa009f-f179-493b-8f45-db2adbd6087f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:46:53.783911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:46:53.810293 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f1511-b3bf-4a5d-996e-b85a6837eeed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:46:53.810903 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f1511-b3bf-4a5d-996e-b85a6837eeed service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:46:53.811181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f1511-b3bf-4a5d-996e-b85a6837eeed service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:46:53.811450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f1511-b3bf-4a5d-996e-b85a6837eeed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:46:53.813077 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-308f1511-b3bf-4a5d-996e-b85a6837eeed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:46:53.925421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f1511-b3bf-4a5d-996e-b85a6837eeed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:46:53.926884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f1511-b3bf-4a5d-996e-b85a6837eeed service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:46:54.060779 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 738/1470] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:46:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:47:03.333479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=22 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:04.058100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=66 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:14.361770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8672e-e6fa-4f66-9c14-946d1f9c3316 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:14.362390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8672e-e6fa-4f66-9c14-946d1f9c3316 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:14.362390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8672e-e6fa-4f66-9c14-946d1f9c3316 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:14.363582 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 733/1471] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:47:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:47:14.369038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f269651-e517-41b8-8e50-bd38ddd90e60 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:14.369571 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f269651-e517-41b8-8e50-bd38ddd90e60 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:14.369571 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f269651-e517-41b8-8e50-bd38ddd90e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:14.442825 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8f269651-e517-41b8-8e50-bd38ddd90e60 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:47:14.444074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8f269651-e517-41b8-8e50-bd38ddd90e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:14.538771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 739/1472] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:47:14.844065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201a43f-285b-4fb1-b7e4-97dcac6585a5 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:14.844377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201a43f-285b-4fb1-b7e4-97dcac6585a5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:14.844653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201a43f-285b-4fb1-b7e4-97dcac6585a5 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:14.846483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 734/1473] 10.222.0.60 () {54 vars in 962 bytes} [Mon Sep 19 14:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:47:14.851417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd499777-a4f6-4ebe-92e3-75aed5891100 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:14.851705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd499777-a4f6-4ebe-92e3-75aed5891100 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:14.852008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd499777-a4f6-4ebe-92e3-75aed5891100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:14.930905 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-dd499777-a4f6-4ebe-92e3-75aed5891100 None None] MFA Rules not processed for user `caab7aaa72634b358d717768fbcb48d8`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:47:14.932136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-dd499777-a4f6-4ebe-92e3-75aed5891100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:14.952287 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:15.007072 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 740/1474] 10.222.0.60 () {58 vars in 1059 bytes} [Mon Sep 19 14:47:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:47:15.014717 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201a43f-285b-4fb1-b7e4-97dcac6585a5 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:47:15.016292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1201a43f-285b-4fb1-b7e4-97dcac6585a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:15.119996 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201a43f-285b-4fb1-b7e4-97dcac6585a5 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:47:15.192700 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c1c83-8c83-4160-a0e3-1140ba503381 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caab7aaa72634b358d717768fbcb48d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:47:15.193287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c1c83-8c83-4160-a0e3-1140ba503381 service glance] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:15.193559 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c1c83-8c83-4160-a0e3-1140ba503381 service glance] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:15.193826 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c1c83-8c83-4160-a0e3-1140ba503381 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:15.195396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4c1c1c83-8c83-4160-a0e3-1140ba503381 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:15.249027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:15.304777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c1c83-8c83-4160-a0e3-1140ba503381 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:47:15.306214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c1c83-8c83-4160-a0e3-1140ba503381 service glance] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:47:15.324641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 735/1475] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:47:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:47:17.694374 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d93617-79e0-471b-b80f-71fc9f78dc12 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:17.694957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d93617-79e0-471b-b80f-71fc9f78dc12 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:17.694957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d93617-79e0-471b-b80f-71fc9f78dc12 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:17.696290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 741/1476] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:47:17.701600 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455fd53-407c-434d-923b-2812ca18c40e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:17.702036 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455fd53-407c-434d-923b-2812ca18c40e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:17.702142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5455fd53-407c-434d-923b-2812ca18c40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:17.772984 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5455fd53-407c-434d-923b-2812ca18c40e None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:47:17.774179 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5455fd53-407c-434d-923b-2812ca18c40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:17.872938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 736/1477] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:47:18.145112 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d93617-79e0-471b-b80f-71fc9f78dc12 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:47:18.146661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c6d93617-79e0-471b-b80f-71fc9f78dc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:18.275309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d93617-79e0-471b-b80f-71fc9f78dc12 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:47:18.324642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:18.369747 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65c9c5-a8c8-4001-b111-617776bd9cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:47:18.370349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65c9c5-a8c8-4001-b111-617776bd9cce admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:18.370622 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65c9c5-a8c8-4001-b111-617776bd9cce admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:18.370886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65c9c5-a8c8-4001-b111-617776bd9cce admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:18.371506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65c9c5-a8c8-4001-b111-617776bd9cce admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:47:18.373266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: WARNING py.warnings [None req-7a65c9c5-a8c8-4001-b111-617776bd9cce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 14:47:18.373266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: warnings.warn(msg) Sep 19 14:47:18.373266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: Sep 19 14:47:18.374373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65c9c5-a8c8-4001-b111-617776bd9cce admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:47:18.393503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 742/1478] 10.222.0.60 () {56 vars in 1108 bytes} [Mon Sep 19 14:47:18 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:47:20.619661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591766c3-58d6-4737-9d99-3133794005c4 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:20.620268 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591766c3-58d6-4737-9d99-3133794005c4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:20.620268 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591766c3-58d6-4737-9d99-3133794005c4 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:20.621616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 737/1479] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:47:20.627163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1ac95-cb02-4216-96a5-591c837f210d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:20.627605 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1ac95-cb02-4216-96a5-591c837f210d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:20.627724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c1ac95-cb02-4216-96a5-591c837f210d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:20.693676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-57c1ac95-cb02-4216-96a5-591c837f210d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:47:20.694887 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-57c1ac95-cb02-4216-96a5-591c837f210d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:20.791210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 743/1480] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:47:21.088078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591766c3-58d6-4737-9d99-3133794005c4 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:47:21.089617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-591766c3-58d6-4737-9d99-3133794005c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:21.099435 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:21.235177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591766c3-58d6-4737-9d99-3133794005c4 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:47:21.314963 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:21.347120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1bfa44-aec5-4406-8ef3-35c22cdf5c00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:47:21.347640 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bfa44-aec5-4406-8ef3-35c22cdf5c00 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:21.347952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bfa44-aec5-4406-8ef3-35c22cdf5c00 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:21.348226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1bfa44-aec5-4406-8ef3-35c22cdf5c00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:21.349816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ac1bfa44-aec5-4406-8ef3-35c22cdf5c00 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:21.438156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bfa44-aec5-4406-8ef3-35c22cdf5c00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:47:21.440391 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1bfa44-aec5-4406-8ef3-35c22cdf5c00 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:47:21.458057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 738/1481] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:47:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:47:23.854682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb31754-0880-4ea4-9b98-fbe05f1cc0c0 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:23.855236 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb31754-0880-4ea4-9b98-fbe05f1cc0c0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:23.855236 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb31754-0880-4ea4-9b98-fbe05f1cc0c0 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:23.856595 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 744/1482] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:47:23.862083 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f44c4a-dd57-4ada-b208-7930eedbf79f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:23.862506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f44c4a-dd57-4ada-b208-7930eedbf79f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:23.862634 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f44c4a-dd57-4ada-b208-7930eedbf79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:23.930876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-88f44c4a-dd57-4ada-b208-7930eedbf79f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:47:23.932128 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-88f44c4a-dd57-4ada-b208-7930eedbf79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:23.978999 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:24.029650 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 739/1483] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:47:24.324227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb31754-0880-4ea4-9b98-fbe05f1cc0c0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:47:24.325755 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-dbb31754-0880-4ea4-9b98-fbe05f1cc0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:24.349477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:24.429985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb31754-0880-4ea4-9b98-fbe05f1cc0c0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:47:24.504000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18276660-0ca8-43ad-9208-bd1bb23fe33f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:47:24.504606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18276660-0ca8-43ad-9208-bd1bb23fe33f service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:24.504881 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18276660-0ca8-43ad-9208-bd1bb23fe33f service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:24.505146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18276660-0ca8-43ad-9208-bd1bb23fe33f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:24.506722 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-18276660-0ca8-43ad-9208-bd1bb23fe33f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:24.582004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:24.618606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18276660-0ca8-43ad-9208-bd1bb23fe33f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:47:24.620101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18276660-0ca8-43ad-9208-bd1bb23fe33f service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:47:24.638555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 745/1484] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:47:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:47:26.872458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a8694-5dec-45f1-84d4-edc8ba6a264c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:26.873049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a8694-5dec-45f1-84d4-edc8ba6a264c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:26.873049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a8694-5dec-45f1-84d4-edc8ba6a264c None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:26.874346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 740/1485] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:47:26.880244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1b380-e8c0-4e4e-adb9-b74fbedbea3f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:26.880703 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1b380-e8c0-4e4e-adb9-b74fbedbea3f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:26.880824 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1b380-e8c0-4e4e-adb9-b74fbedbea3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:26.946746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-18f1b380-e8c0-4e4e-adb9-b74fbedbea3f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:47:26.947988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-18f1b380-e8c0-4e4e-adb9-b74fbedbea3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:27.044111 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 746/1486] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:47:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:47:27.332628 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a8694-5dec-45f1-84d4-edc8ba6a264c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:47:27.334160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4d8a8694-5dec-45f1-84d4-edc8ba6a264c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:27.436495 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a8694-5dec-45f1-84d4-edc8ba6a264c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:47:27.513398 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:27.514985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f78400-0188-4daa-b590-ca90bbc77d73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:47:27.515655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f78400-0188-4daa-b590-ca90bbc77d73 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:27.516153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f78400-0188-4daa-b590-ca90bbc77d73 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:27.516551 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f78400-0188-4daa-b590-ca90bbc77d73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:27.518508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e2f78400-0188-4daa-b590-ca90bbc77d73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:27.627254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f78400-0188-4daa-b590-ca90bbc77d73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:47:27.628710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f78400-0188-4daa-b590-ca90bbc77d73 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:47:27.646611 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 741/1487] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:47:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:47:29.987035 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc939af3-32fe-44ef-9971-2db4801c07f2 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:29.987637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc939af3-32fe-44ef-9971-2db4801c07f2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:29.987637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc939af3-32fe-44ef-9971-2db4801c07f2 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:29.988940 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 747/1488] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:47:29.994522 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42410b4d-0fdf-45ca-9ed4-7d51703eca4e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:29.994937 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42410b4d-0fdf-45ca-9ed4-7d51703eca4e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:29.995038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42410b4d-0fdf-45ca-9ed4-7d51703eca4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:30.065361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-42410b4d-0fdf-45ca-9ed4-7d51703eca4e None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:47:30.066557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-42410b4d-0fdf-45ca-9ed4-7d51703eca4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:30.121202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:30.166117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 742/1489] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:47:30.457352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc939af3-32fe-44ef-9971-2db4801c07f2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:47:30.458905 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fc939af3-32fe-44ef-9971-2db4801c07f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:30.495800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:30.565847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc939af3-32fe-44ef-9971-2db4801c07f2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:47:30.641037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c7fde3-1d61-4573-a0d7-74aea11f9f8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:47:30.642249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7fde3-1d61-4573-a0d7-74aea11f9f8f service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:30.642534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7fde3-1d61-4573-a0d7-74aea11f9f8f service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:30.642886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c7fde3-1d61-4573-a0d7-74aea11f9f8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:30.644551 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-53c7fde3-1d61-4573-a0d7-74aea11f9f8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:30.725798 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:30.755148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7fde3-1d61-4573-a0d7-74aea11f9f8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:47:30.756608 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c7fde3-1d61-4573-a0d7-74aea11f9f8f service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:47:30.774868 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 748/1490] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:47:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:47:33.166886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcdc89-7dd2-483a-b488-7e1baf0b8b34 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:33.167424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcdc89-7dd2-483a-b488-7e1baf0b8b34 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:33.167424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcdc89-7dd2-483a-b488-7e1baf0b8b34 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:33.168805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 743/1491] 10.222.0.60 () {54 vars in 927 bytes} [Mon Sep 19 14:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:47:33.174645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62a406-efb0-43a4-a195-73f963e66a3c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:33.175090 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62a406-efb0-43a4-a195-73f963e66a3c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:33.175196 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c62a406-efb0-43a4-a195-73f963e66a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:33.242499 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-6c62a406-efb0-43a4-a195-73f963e66a3c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:47:33.243728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6c62a406-efb0-43a4-a195-73f963e66a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:33.340017 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 749/1492] 10.222.0.60 () {58 vars in 1024 bytes} [Mon Sep 19 14:47:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:47:33.627520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dcdc89-7dd2-483a-b488-7e1baf0b8b34 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:47:33.629085 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-98dcdc89-7dd2-483a-b488-7e1baf0b8b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:33.736245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dcdc89-7dd2-483a-b488-7e1baf0b8b34 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:47:33.809238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a4995-1d8a-495c-ac6b-82af70dc8b90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:47:33.809838 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a4995-1d8a-495c-ac6b-82af70dc8b90 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:47:33.810123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a4995-1d8a-495c-ac6b-82af70dc8b90 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:47:33.810398 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a4995-1d8a-495c-ac6b-82af70dc8b90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:47:33.812031 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f35a4995-1d8a-495c-ac6b-82af70dc8b90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:47:33.821126 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:33.924289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a4995-1d8a-495c-ac6b-82af70dc8b90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:47:33.925767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35a4995-1d8a-495c-ac6b-82af70dc8b90 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:47:33.944210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 744/1493] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:47:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:47:43.338963 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=82 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:47:43.941957 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=50 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:48:48.119224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b896af-ce91-4151-b9a4-eee31ed25722 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:48.119790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b896af-ce91-4151-b9a4-eee31ed25722 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:48.119790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b896af-ce91-4151-b9a4-eee31ed25722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:48.196404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-62b896af-ce91-4151-b9a4-eee31ed25722 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:48:48.197606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-62b896af-ce91-4151-b9a4-eee31ed25722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:48.295674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 750/1494] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:48:48.303596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca862a-a64e-4deb-9d6a-f94f8a3c5181 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:48.304155 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca862a-a64e-4deb-9d6a-f94f8a3c5181 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:48.304360 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca862a-a64e-4deb-9d6a-f94f8a3c5181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:48.383652 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2eca862a-a64e-4deb-9d6a-f94f8a3c5181 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:48:48.384902 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2eca862a-a64e-4deb-9d6a-f94f8a3c5181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:48.480069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 745/1495] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:48:48.484808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b896af-ce91-4151-b9a4-eee31ed25722 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:48:48.486024 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-62b896af-ce91-4151-b9a4-eee31ed25722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:48.497513 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b896af-ce91-4151-b9a4-eee31ed25722 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:48:48.561396 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:48:48.594875 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b754ca3-b8f4-4027-acd0-dee6c405fe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:48:48.595472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b754ca3-b8f4-4027-acd0-dee6c405fe4d admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:48.595745 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b754ca3-b8f4-4027-acd0-dee6c405fe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:48.596040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b754ca3-b8f4-4027-acd0-dee6c405fe4d admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:48.596671 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b754ca3-b8f4-4027-acd0-dee6c405fe4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:48:48.596968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b754ca3-b8f4-4027-acd0-dee6c405fe4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:48:48.597968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: WARNING py.warnings [None req-2b754ca3-b8f4-4027-acd0-dee6c405fe4d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 14:48:48.597968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: warnings.warn(msg) Sep 19 14:48:48.597968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: Sep 19 14:48:48.598834 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b754ca3-b8f4-4027-acd0-dee6c405fe4d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:48:48.610818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 751/1496] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:48:48.619171 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca862a-a64e-4deb-9d6a-f94f8a3c5181 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:48:48.631533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca862a-a64e-4deb-9d6a-f94f8a3c5181 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:48:48.694050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:48:48.727300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb070d-7a96-4d39-a3fc-ca6ad753a6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:48:48.727938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb070d-7a96-4d39-a3fc-ca6ad753a6f0 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:48.728228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb070d-7a96-4d39-a3fc-ca6ad753a6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:48.728493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb070d-7a96-4d39-a3fc-ca6ad753a6f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:48.729109 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb070d-7a96-4d39-a3fc-ca6ad753a6f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:48:48.730854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: WARNING py.warnings [None req-25cb070d-7a96-4d39-a3fc-ca6ad753a6f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 14:48:48.730854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: warnings.warn(msg) Sep 19 14:48:48.730854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: Sep 19 14:48:48.732073 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb070d-7a96-4d39-a3fc-ca6ad753a6f0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:48:48.787451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 746/1497] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:48:48 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:48:48.791556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b754ca3-b8f4-4027-acd0-dee6c405fe4d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:48:48.793266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2b754ca3-b8f4-4027-acd0-dee6c405fe4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:48.832521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b754ca3-b8f4-4027-acd0-dee6c405fe4d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:48:48.931801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c586b03-54d9-4981-a42f-838b8c1ce3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:48:48.932434 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c586b03-54d9-4981-a42f-838b8c1ce3ff admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:48.932713 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c586b03-54d9-4981-a42f-838b8c1ce3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:48.932984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c586b03-54d9-4981-a42f-838b8c1ce3ff admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:48.933625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c586b03-54d9-4981-a42f-838b8c1ce3ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:48:48.934631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: WARNING py.warnings [None req-4c586b03-54d9-4981-a42f-838b8c1ce3ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 14:48:48.934631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: warnings.warn(msg) Sep 19 14:48:48.934631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: Sep 19 14:48:48.936475 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c586b03-54d9-4981-a42f-838b8c1ce3ff admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:48:48.978379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 752/1498] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:48:48 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:48:48.983105 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb070d-7a96-4d39-a3fc-ca6ad753a6f0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:48:48.984634 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-25cb070d-7a96-4d39-a3fc-ca6ad753a6f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:49.046626 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb070d-7a96-4d39-a3fc-ca6ad753a6f0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:48:49.132071 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:48:49.143196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11e5a8-1f53-4f0b-82a8-440525473498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:48:49.143968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11e5a8-1f53-4f0b-82a8-440525473498 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:49.144253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11e5a8-1f53-4f0b-82a8-440525473498 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:49.144527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11e5a8-1f53-4f0b-82a8-440525473498 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:49.145158 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11e5a8-1f53-4f0b-82a8-440525473498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:48:49.146182 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: WARNING py.warnings [None req-dc11e5a8-1f53-4f0b-82a8-440525473498 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 14:48:49.146182 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: warnings.warn(msg) Sep 19 14:48:49.146182 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: Sep 19 14:48:49.147063 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11e5a8-1f53-4f0b-82a8-440525473498 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:48:49.159437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 747/1499] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:48:48 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:48:49.163535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c586b03-54d9-4981-a42f-838b8c1ce3ff admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:48:49.165143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4c586b03-54d9-4981-a42f-838b8c1ce3ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:49.177075 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c586b03-54d9-4981-a42f-838b8c1ce3ff admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:48:49.202639 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:48:49.273672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc7420-e936-4f0e-801f-69bbeca92f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:48:49.274274 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc7420-e936-4f0e-801f-69bbeca92f37 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:49.274546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc7420-e936-4f0e-801f-69bbeca92f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:49.274810 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc7420-e936-4f0e-801f-69bbeca92f37 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:49.275434 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc7420-e936-4f0e-801f-69bbeca92f37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:48:49.276457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: WARNING py.warnings [None req-97bc7420-e936-4f0e-801f-69bbeca92f37 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 14:48:49.276457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: warnings.warn(msg) Sep 19 14:48:49.276457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: Sep 19 14:48:49.277333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc7420-e936-4f0e-801f-69bbeca92f37 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:48:49.289415 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 753/1500] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:48:49.293612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11e5a8-1f53-4f0b-82a8-440525473498 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:48:49.295245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dc11e5a8-1f53-4f0b-82a8-440525473498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:49.331351 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11e5a8-1f53-4f0b-82a8-440525473498 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:48:49.446968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e3a30-5eb4-4e61-9f84-33cbe25b0c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:48:49.447624 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e3a30-5eb4-4e61-9f84-33cbe25b0c75 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:49.448208 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e3a30-5eb4-4e61-9f84-33cbe25b0c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:49.448484 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e3a30-5eb4-4e61-9f84-33cbe25b0c75 admin admin] PATH_INFO: `/v3/projects/5af36b267a11447280993ded420cd980/users/2b1256d6cfd1472f9bf11881ddb87ebd/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:49.471108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e3a30-5eb4-4e61-9f84-33cbe25b0c75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5af36b267a11447280993ded420cd980, user_id=2b1256d6cfd1472f9bf11881ddb87ebd, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:48:49.472173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: WARNING py.warnings [None req-db2e3a30-5eb4-4e61-9f84-33cbe25b0c75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 14:48:49.472173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: warnings.warn(msg) Sep 19 14:48:49.472173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: Sep 19 14:48:49.474095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e3a30-5eb4-4e61-9f84-33cbe25b0c75 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:48:49.498015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 748/1501] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:48:49 2022] PUT /identity/v3/projects/5af36b267a11447280993ded420cd980/users/2b1256d6cfd1472f9bf11881ddb87ebd/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:48:49.510801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d0d15-f586-424d-b65a-0cc3cadefcba None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:49.511086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d0d15-f586-424d-b65a-0cc3cadefcba None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:49.511359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d0d15-f586-424d-b65a-0cc3cadefcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:49.566928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f49d0d15-f586-424d-b65a-0cc3cadefcba None None] MFA Rules not processed for user `2b1256d6cfd1472f9bf11881ddb87ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:48:49.568157 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f49d0d15-f586-424d-b65a-0cc3cadefcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:49.644422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 754/1502] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:48:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:48:49.658142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c4d88-d78a-442a-b232-78fdbf4c76af None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:49.658425 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c4d88-d78a-442a-b232-78fdbf4c76af None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:49.658699 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884c4d88-d78a-442a-b232-78fdbf4c76af None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:49.660054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 749/1503] 10.222.0.60 () {54 vars in 962 bytes} [Mon Sep 19 14:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:48:49.664782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192cea4-b4f5-4b27-adc7-58e1a57d6117 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:49.665068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192cea4-b4f5-4b27-adc7-58e1a57d6117 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:49.665342 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9192cea4-b4f5-4b27-adc7-58e1a57d6117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:49.699264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:48:49.747783 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9192cea4-b4f5-4b27-adc7-58e1a57d6117 None None] MFA Rules not processed for user `006fe9ba3b0a40f29fa2be7620e548c4`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:48:49.749589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9192cea4-b4f5-4b27-adc7-58e1a57d6117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:49.830039 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 755/1504] 10.222.0.60 () {58 vars in 1059 bytes} [Mon Sep 19 14:48:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:48:49.839634 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c4d88-d78a-442a-b232-78fdbf4c76af None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:48:49.841184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-884c4d88-d78a-442a-b232-78fdbf4c76af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:49.895415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:48:49.950795 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884c4d88-d78a-442a-b232-78fdbf4c76af None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:48:50.023992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21ead4-e531-4478-aa38-00d857af1e4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:48:50.024643 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21ead4-e531-4478-aa38-00d857af1e4d service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:50.024965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21ead4-e531-4478-aa38-00d857af1e4d service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:50.025260 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21ead4-e531-4478-aa38-00d857af1e4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:50.026851 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9f21ead4-e531-4478-aa38-00d857af1e4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:50.148225 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21ead4-e531-4478-aa38-00d857af1e4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:48:50.149693 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21ead4-e531-4478-aa38-00d857af1e4d service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:48:50.153113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:48:50.168536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 750/1505] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:48:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:48:50.214177 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f6f32-e1af-4fa3-a483-d090e47cf821 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:50.214635 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f6f32-e1af-4fa3-a483-d090e47cf821 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:50.214759 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f6f32-e1af-4fa3-a483-d090e47cf821 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:50.215973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 756/1506] 10.222.0.60 () {52 vars in 830 bytes} [Mon Sep 19 14:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:48:50.220333 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21ead4-e531-4478-aa38-00d857af1e4d service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:48:50.221558 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9f21ead4-e531-4478-aa38-00d857af1e4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:50.317411 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21ead4-e531-4478-aa38-00d857af1e4d service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:48:50.415001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d87c5-da3c-4f1b-8053-3145b208e2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:48:50.415705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d87c5-da3c-4f1b-8053-3145b208e2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:50.419776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d87c5-da3c-4f1b-8053-3145b208e2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:50.420107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d87c5-da3c-4f1b-8053-3145b208e2b2 admin admin] PATH_INFO: `/v3/users/2b1256d6cfd1472f9bf11881ddb87ebd` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:50.436415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d87c5-da3c-4f1b-8053-3145b208e2b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b1256d6cfd1472f9bf11881ddb87ebd)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:48:50.438056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: WARNING py.warnings [None req-334d87c5-da3c-4f1b-8053-3145b208e2b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 14:48:50.438056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: warnings.warn(msg) Sep 19 14:48:50.438056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: Sep 19 14:48:50.439287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d87c5-da3c-4f1b-8053-3145b208e2b2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:48:50.452155 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:48:50.508658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-334d87c5-da3c-4f1b-8053-3145b208e2b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b1256d6cfd1472f9bf11881ddb87ebd'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:48:50.523244 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-334d87c5-da3c-4f1b-8053-3145b208e2b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b1256d6cfd1472f9bf11881ddb87ebd'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:48:50.546237 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-334d87c5-da3c-4f1b-8053-3145b208e2b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b1256d6cfd1472f9bf11881ddb87ebd'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:48:50.556180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-334d87c5-da3c-4f1b-8053-3145b208e2b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b1256d6cfd1472f9bf11881ddb87ebd'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:48:50.556859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-334d87c5-da3c-4f1b-8053-3145b208e2b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b1256d6cfd1472f9bf11881ddb87ebd'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:48:50.562571 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 751/1507] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:48:50 2022] DELETE /identity/v3/users/2b1256d6cfd1472f9bf11881ddb87ebd => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:48:50.567210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb509937-6e2c-4139-a8cb-6831c6beeeb8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:50.567529 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb509937-6e2c-4139-a8cb-6831c6beeeb8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:50.567805 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb509937-6e2c-4139-a8cb-6831c6beeeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:50.626344 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-cb509937-6e2c-4139-a8cb-6831c6beeeb8 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:48:50.627536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cb509937-6e2c-4139-a8cb-6831c6beeeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:50.707068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:48:50.720924 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 757/1508] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:48:50.732330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d87c5-da3c-4f1b-8053-3145b208e2b2 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:48:50.734301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-334d87c5-da3c-4f1b-8053-3145b208e2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:50.856763 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d87c5-da3c-4f1b-8053-3145b208e2b2 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:48:50.915695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:48:50.936823 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d904e85-2ef6-49d2-a8ab-c1f56ca2e948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:48:50.937438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d904e85-2ef6-49d2-a8ab-c1f56ca2e948 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:50.937720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d904e85-2ef6-49d2-a8ab-c1f56ca2e948 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:50.938001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d904e85-2ef6-49d2-a8ab-c1f56ca2e948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:50.939618 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4d904e85-2ef6-49d2-a8ab-c1f56ca2e948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:51.059906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d904e85-2ef6-49d2-a8ab-c1f56ca2e948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:48:51.061297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d904e85-2ef6-49d2-a8ab-c1f56ca2e948 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:48:51.084485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 752/1509] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:48:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:48:51.422605 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb509937-6e2c-4139-a8cb-6831c6beeeb8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:48:51.423883 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cb509937-6e2c-4139-a8cb-6831c6beeeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:48:51.435520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb509937-6e2c-4139-a8cb-6831c6beeeb8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:48:51.535663 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:48:51.536324 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:48:51.536603 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:48:51.536879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0 admin admin] PATH_INFO: `/v3/projects/5af36b267a11447280993ded420cd980` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:48:51.547191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5af36b267a11447280993ded420cd980)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:48:51.548394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: WARNING py.warnings [None req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 14:48:51.548394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: warnings.warn(msg) Sep 19 14:48:51.548394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: Sep 19 14:48:51.549759 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:48:51.610098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5af36b267a11447280993ded420cd980', 'request_id': 'req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:48:51.616055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5af36b267a11447280993ded420cd980', 'request_id': 'req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:48:51.629858 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5af36b267a11447280993ded420cd980', 'request_id': 'req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:48:51.656564 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0 admin admin] The token cache is being invalidate because project 5af36b267a11447280993ded420cd980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:48:51.656855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:48:51.657378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca22f2a3-6f11-4e45-9e80-70cd95ee11b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:48:51.658681 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 758/1510] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:48:51 2022] DELETE /identity/v3/projects/5af36b267a11447280993ded420cd980 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:49:01.081550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=62 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:49:01.653839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:50:20.951164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f93ca1-49af-49f6-97a1-dcaec8cfe6de None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:20.951798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f93ca1-49af-49f6-97a1-dcaec8cfe6de None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:20.951798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f93ca1-49af-49f6-97a1-dcaec8cfe6de None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:20.953151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 14:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:50:20.958450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bdea9-1f31-49ea-a019-417fcd39c03f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:20.958936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bdea9-1f31-49ea-a019-417fcd39c03f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:20.959057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bdea9-1f31-49ea-a019-417fcd39c03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:21.039205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-815bdea9-1f31-49ea-a019-417fcd39c03f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:50:21.040440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-815bdea9-1f31-49ea-a019-417fcd39c03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:50:21.159145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 14:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:50:21.315772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f93ca1-49af-49f6-97a1-dcaec8cfe6de None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:50:21.318173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e1f93ca1-49af-49f6-97a1-dcaec8cfe6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:50:21.436426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f93ca1-49af-49f6-97a1-dcaec8cfe6de None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:50:21.441951 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7d37d-20b7-4264-9762-df168cd07436 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:50:21.442556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7d37d-20b7-4264-9762-df168cd07436 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:21.442840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7d37d-20b7-4264-9762-df168cd07436 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:21.443117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7d37d-20b7-4264-9762-df168cd07436 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:21.451582 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7d37d-20b7-4264-9762-df168cd07436 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:50:21.453213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7d37d-20b7-4264-9762-df168cd07436 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:50:21.462589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 754/1513] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:50:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:50:22.565009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66699a-bcba-45ae-9783-23e0cdc59d44 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:22.565586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66699a-bcba-45ae-9783-23e0cdc59d44 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:22.565586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66699a-bcba-45ae-9783-23e0cdc59d44 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:22.566886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 14:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:50:22.573155 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a4216-0c46-4801-9a0b-cc908ef96795 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:22.573598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a4216-0c46-4801-9a0b-cc908ef96795 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:22.573700 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a4216-0c46-4801-9a0b-cc908ef96795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:22.594177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9c2a4216-0c46-4801-9a0b-cc908ef96795 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:50:22.595383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9c2a4216-0c46-4801-9a0b-cc908ef96795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:50:22.603237 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 14:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:50:22.760930 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66699a-bcba-45ae-9783-23e0cdc59d44 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:50:22.782108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66699a-bcba-45ae-9783-23e0cdc59d44 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:50:22.787909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa580d9e-dbe6-456d-a95f-ad3026ded9fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:50:22.788547 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa580d9e-dbe6-456d-a95f-ad3026ded9fb service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:22.788827 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa580d9e-dbe6-456d-a95f-ad3026ded9fb service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:22.789149 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa580d9e-dbe6-456d-a95f-ad3026ded9fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:22.797787 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa580d9e-dbe6-456d-a95f-ad3026ded9fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:50:22.799402 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa580d9e-dbe6-456d-a95f-ad3026ded9fb service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:50:22.811321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 761/1516] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:50:23.212810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be7824-acc0-4d27-832d-04ba9b90672c None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:23.213095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be7824-acc0-4d27-832d-04ba9b90672c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:23.213379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be7824-acc0-4d27-832d-04ba9b90672c None None] PATH_INFO: `/v3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:23.214738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 756/1517] 10.222.0.71 () {54 vars in 925 bytes} [Mon Sep 19 14:50:23 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:50:23.221606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911d7df-bf09-4f01-b796-23c169073789 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:23.221895 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911d7df-bf09-4f01-b796-23c169073789 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:23.222173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911d7df-bf09-4f01-b796-23c169073789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:23.282254 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c911d7df-bf09-4f01-b796-23c169073789 None None] MFA Rules not processed for user `568189358b154b66a729fae2f0411d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:50:23.283521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c911d7df-bf09-4f01-b796-23c169073789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:50:23.301511 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:50:23.357775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 762/1518] 10.222.0.71 () {58 vars in 1015 bytes} [Mon Sep 19 14:50:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:50:26.940237 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9ab93-7054-46ee-abcb-ef8713ede2e3 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:26.940680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9ab93-7054-46ee-abcb-ef8713ede2e3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:26.940826 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9ab93-7054-46ee-abcb-ef8713ede2e3 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:26.942183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 757/1519] 10.222.0.60 () {54 vars in 961 bytes} [Mon Sep 19 14:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:50:26.950762 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d46ec-aff3-4621-a820-e62c6f34d788 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:26.951233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d46ec-aff3-4621-a820-e62c6f34d788 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:26.951341 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d46ec-aff3-4621-a820-e62c6f34d788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:26.971712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a62d46ec-aff3-4621-a820-e62c6f34d788 None None] MFA Rules not processed for user `568189358b154b66a729fae2f0411d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:50:26.974250 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a62d46ec-aff3-4621-a820-e62c6f34d788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:50:26.983648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 763/1520] 10.222.0.60 () {58 vars in 1058 bytes} [Mon Sep 19 14:50:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:50:26.991447 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9ab93-7054-46ee-abcb-ef8713ede2e3 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:50:27.006084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9ab93-7054-46ee-abcb-ef8713ede2e3 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:50:27.011154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70a362-3144-4449-9c82-84c541652c7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '568189358b154b66a729fae2f0411d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:50:27.011756 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70a362-3144-4449-9c82-84c541652c7a service placement] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:27.012100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70a362-3144-4449-9c82-84c541652c7a service placement] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:27.012374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70a362-3144-4449-9c82-84c541652c7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:27.020721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c70a362-3144-4449-9c82-84c541652c7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:50:27.022305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c70a362-3144-4449-9c82-84c541652c7a service placement] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:50:27.031951 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 758/1521] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:50:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:50:34.297960 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb50cc-2781-446a-abdb-06eb502becfe None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:34.298534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb50cc-2781-446a-abdb-06eb502becfe None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:34.298534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb50cc-2781-446a-abdb-06eb502becfe None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:34.299869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 14:50:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:50:34.306458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce490da-7d65-4544-a4fe-182e664d8c04 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:34.306906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce490da-7d65-4544-a4fe-182e664d8c04 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:34.307012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce490da-7d65-4544-a4fe-182e664d8c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:34.326766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6ce490da-7d65-4544-a4fe-182e664d8c04 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:50:34.327998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6ce490da-7d65-4544-a4fe-182e664d8c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:50:34.335811 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 14:50:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:50:34.490472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb50cc-2781-446a-abdb-06eb502becfe None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:50:34.505057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb50cc-2781-446a-abdb-06eb502becfe None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:50:34.510517 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0753028-daed-4037-ba62-c3461d65f4b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:50:34.511113 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0753028-daed-4037-ba62-c3461d65f4b9 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:34.511396 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0753028-daed-4037-ba62-c3461d65f4b9 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:34.511673 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0753028-daed-4037-ba62-c3461d65f4b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:34.520300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0753028-daed-4037-ba62-c3461d65f4b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:50:34.521911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0753028-daed-4037-ba62-c3461d65f4b9 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:50:34.532191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 765/1524] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:50:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:50:35.604874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb88c1-906b-45d4-b219-9857e9934a99 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:35.605469 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb88c1-906b-45d4-b219-9857e9934a99 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:35.605570 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb88c1-906b-45d4-b219-9857e9934a99 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:35.607489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 19 14:50:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:50:35.614555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2a2b8-0a4d-40df-94ec-a14d048402d1 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:35.615044 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2a2b8-0a4d-40df-94ec-a14d048402d1 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:35.615174 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2a2b8-0a4d-40df-94ec-a14d048402d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:35.636139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-14e2a2b8-0a4d-40df-94ec-a14d048402d1 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:50:35.637354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-14e2a2b8-0a4d-40df-94ec-a14d048402d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:50:35.645623 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 19 14:50:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:50:35.796070 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb88c1-906b-45d4-b219-9857e9934a99 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:50:35.810584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb88c1-906b-45d4-b219-9857e9934a99 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:50:35.816057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f19efd5-096d-4a58-9cc7-2360634d2c0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:50:35.816657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19efd5-096d-4a58-9cc7-2360634d2c0a service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:35.816942 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19efd5-096d-4a58-9cc7-2360634d2c0a service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:35.817222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f19efd5-096d-4a58-9cc7-2360634d2c0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:35.825845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19efd5-096d-4a58-9cc7-2360634d2c0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:50:35.827427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f19efd5-096d-4a58-9cc7-2360634d2c0a service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:50:35.836719 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 761/1527] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:50:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:50:39.931058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b03e4-7b2e-467a-be71-acf1834f2bb2 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:39.931637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b03e4-7b2e-467a-be71-acf1834f2bb2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:39.931637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b03e4-7b2e-467a-be71-acf1834f2bb2 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:39.933048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 14:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:50:39.938409 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a9b7c-5315-47bf-a707-5d9723ebc146 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:39.938863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a9b7c-5315-47bf-a707-5d9723ebc146 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:39.938982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a9b7c-5315-47bf-a707-5d9723ebc146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:39.957863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-333a9b7c-5315-47bf-a707-5d9723ebc146 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:50:39.959037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-333a9b7c-5315-47bf-a707-5d9723ebc146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:50:39.967236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 14:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:50:40.260727 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b03e4-7b2e-467a-be71-acf1834f2bb2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:50:40.262462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-725b03e4-7b2e-467a-be71-acf1834f2bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:50:40.343481 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b03e4-7b2e-467a-be71-acf1834f2bb2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:50:40.347513 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057e9c3-8049-4775-a3a0-bd522f96b91a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:50:40.348028 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057e9c3-8049-4775-a3a0-bd522f96b91a service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:40.348284 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057e9c3-8049-4775-a3a0-bd522f96b91a service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:40.348558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057e9c3-8049-4775-a3a0-bd522f96b91a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:40.358103 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057e9c3-8049-4775-a3a0-bd522f96b91a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:50:40.359526 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057e9c3-8049-4775-a3a0-bd522f96b91a service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:50:40.369074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 768/1530] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:50:42.516356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0b488-784e-45a7-a20a-69cddfc71053 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:42.516903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0b488-784e-45a7-a20a-69cddfc71053 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:42.516903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0b488-784e-45a7-a20a-69cddfc71053 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:42.518240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 14:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:50:42.523440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caf433-6b37-4340-8284-8e6685604626 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:42.523910 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caf433-6b37-4340-8284-8e6685604626 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:42.524096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8caf433-6b37-4340-8284-8e6685604626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:42.544981 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c8caf433-6b37-4340-8284-8e6685604626 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:50:42.546273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c8caf433-6b37-4340-8284-8e6685604626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:50:42.554228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 14:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:50:42.854503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0b488-784e-45a7-a20a-69cddfc71053 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:50:42.857083 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e1d0b488-784e-45a7-a20a-69cddfc71053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:50:42.870210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0b488-784e-45a7-a20a-69cddfc71053 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:50:42.873838 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7518e2-d6c2-4c17-a31b-6b6a42562d51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:50:42.874455 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7518e2-d6c2-4c17-a31b-6b6a42562d51 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:42.874739 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7518e2-d6c2-4c17-a31b-6b6a42562d51 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:42.875022 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7518e2-d6c2-4c17-a31b-6b6a42562d51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:42.884987 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7518e2-d6c2-4c17-a31b-6b6a42562d51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:50:42.886451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7518e2-d6c2-4c17-a31b-6b6a42562d51 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:50:42.897194 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 764/1533] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:50:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:50:46.586055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae4f90-0132-42a1-a3dc-c3f05f037e2b None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:46.586629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae4f90-0132-42a1-a3dc-c3f05f037e2b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:46.586629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae4f90-0132-42a1-a3dc-c3f05f037e2b None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:46.587964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 14:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:50:46.593331 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80f468-933b-4ef4-bdfc-c5f5d5cac0de None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:46.593802 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80f468-933b-4ef4-bdfc-c5f5d5cac0de None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:46.593901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80f468-933b-4ef4-bdfc-c5f5d5cac0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:46.612956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-cb80f468-933b-4ef4-bdfc-c5f5d5cac0de None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:50:46.614429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cb80f468-933b-4ef4-bdfc-c5f5d5cac0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:50:46.622687 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 14:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:50:46.927071 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae4f90-0132-42a1-a3dc-c3f05f037e2b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:50:46.928921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-80ae4f90-0132-42a1-a3dc-c3f05f037e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:50:46.993328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:50:47.007889 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae4f90-0132-42a1-a3dc-c3f05f037e2b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:50:47.011425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d31c9-ae99-459e-b3ca-8c39101b7d2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:50:47.012066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d31c9-ae99-459e-b3ca-8c39101b7d2b service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:47.012425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d31c9-ae99-459e-b3ca-8c39101b7d2b service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:47.012697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d31c9-ae99-459e-b3ca-8c39101b7d2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:47.021775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d31c9-ae99-459e-b3ca-8c39101b7d2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:50:47.023205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d31c9-ae99-459e-b3ca-8c39101b7d2b service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:50:47.032567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 771/1536] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:50:49.206640 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6bae9-5d6d-49ae-9ea2-665fe613ce8a None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:49.207194 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6bae9-5d6d-49ae-9ea2-665fe613ce8a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:49.207194 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6bae9-5d6d-49ae-9ea2-665fe613ce8a None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:49.208540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 19 14:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:50:49.213690 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa769208-e8eb-4386-975e-909c0193f25d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:49.214091 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa769208-e8eb-4386-975e-909c0193f25d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:49.214245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa769208-e8eb-4386-975e-909c0193f25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:49.233996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-aa769208-e8eb-4386-975e-909c0193f25d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:50:49.235190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-aa769208-e8eb-4386-975e-909c0193f25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:50:49.243219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 19 14:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:50:49.551195 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6bae9-5d6d-49ae-9ea2-665fe613ce8a None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:50:49.552931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5de6bae9-5d6d-49ae-9ea2-665fe613ce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:50:49.566589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6bae9-5d6d-49ae-9ea2-665fe613ce8a None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:50:49.570177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052eb5e7-5948-4893-8ab4-0387cfb3683e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:50:49.570768 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052eb5e7-5948-4893-8ab4-0387cfb3683e service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:50:49.571050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052eb5e7-5948-4893-8ab4-0387cfb3683e service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:50:49.571324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052eb5e7-5948-4893-8ab4-0387cfb3683e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:50:49.580895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052eb5e7-5948-4893-8ab4-0387cfb3683e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:50:49.582322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052eb5e7-5948-4893-8ab4-0387cfb3683e service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:50:49.591681 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 767/1539] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:50:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:50:59.229094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=11 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:50:59.587537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:13.826644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74db2e5-bf1a-4032-9e32-c53d1346da83 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:13.827318 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74db2e5-bf1a-4032-9e32-c53d1346da83 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:13.827318 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74db2e5-bf1a-4032-9e32-c53d1346da83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:13.856468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f74db2e5-bf1a-4032-9e32-c53d1346da83 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:13.857775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f74db2e5-bf1a-4032-9e32-c53d1346da83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:13.867228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 773/1540] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:13.876602 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc543c-065d-496a-a88f-8d8bd81db505 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:13.877178 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc543c-065d-496a-a88f-8d8bd81db505 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:13.877178 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc543c-065d-496a-a88f-8d8bd81db505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:13.909145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5dfc543c-065d-496a-a88f-8d8bd81db505 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:13.910337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5dfc543c-065d-496a-a88f-8d8bd81db505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:13.925537 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9679ae-de16-4e63-914b-1362127b87e2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:13.925910 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9679ae-de16-4e63-914b-1362127b87e2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:13.926017 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 768/1541] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:13.926111 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9679ae-de16-4e63-914b-1362127b87e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:13.930315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc543c-065d-496a-a88f-8d8bd81db505 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:13.942332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc543c-065d-496a-a88f-8d8bd81db505 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:13.949915 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea66cae-2c45-4175-975a-f483acefabf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:13.950603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea66cae-2c45-4175-975a-f483acefabf2 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:13.950885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea66cae-2c45-4175-975a-f483acefabf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:13.951205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea66cae-2c45-4175-975a-f483acefabf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:13.951894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea66cae-2c45-4175-975a-f483acefabf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:52:13.952203 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea66cae-2c45-4175-975a-f483acefabf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:13.953310 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: WARNING py.warnings [None req-5ea66cae-2c45-4175-975a-f483acefabf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 19 14:52:13.953310 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: warnings.warn(msg) Sep 19 14:52:13.953310 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: Sep 19 14:52:13.954208 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea66cae-2c45-4175-975a-f483acefabf2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:13.964976 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-7f9679ae-de16-4e63-914b-1362127b87e2 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:13.966282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7f9679ae-de16-4e63-914b-1362127b87e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:13.976083 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 769/1542] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:52:13.981527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea66cae-2c45-4175-975a-f483acefabf2 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:13.984426 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 774/1543] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:52:13.994103 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168a914-fd34-4928-a62c-7547c9757096 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:13.994445 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168a914-fd34-4928-a62c-7547c9757096 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:13.994670 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168a914-fd34-4928-a62c-7547c9757096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:13.997264 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea66cae-2c45-4175-975a-f483acefabf2 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.012075 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cc9c7-8418-45a0-8ff6-4164299cda5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.012706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cc9c7-8418-45a0-8ff6-4164299cda5a admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.013001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cc9c7-8418-45a0-8ff6-4164299cda5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.013251 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cc9c7-8418-45a0-8ff6-4164299cda5a admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.013886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0cc9c7-8418-45a0-8ff6-4164299cda5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.015869 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0cc9c7-8418-45a0-8ff6-4164299cda5a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.033124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-6168a914-fd34-4928-a62c-7547c9757096 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:14.034547 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6168a914-fd34-4928-a62c-7547c9757096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:14.048288 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 775/1544] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:14.053582 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168a914-fd34-4928-a62c-7547c9757096 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.070172 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168a914-fd34-4928-a62c-7547c9757096 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.085148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bf049-4683-427a-9eab-a60bfbf335f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.085726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bf049-4683-427a-9eab-a60bfbf335f0 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.086013 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bf049-4683-427a-9eab-a60bfbf335f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.086290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bf049-4683-427a-9eab-a60bfbf335f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.086993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bf049-4683-427a-9eab-a60bfbf335f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:52:14.087519 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bf049-4683-427a-9eab-a60bfbf335f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.092566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bf049-4683-427a-9eab-a60bfbf335f0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.118016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 770/1545] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:13 2022] POST /identity/v3/projects => generated 387 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:52:14.122299 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cc9c7-8418-45a0-8ff6-4164299cda5a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.124377 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 776/1546] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:52:14.126609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bf049-4683-427a-9eab-a60bfbf335f0 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.139872 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bf049-4683-427a-9eab-a60bfbf335f0 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.224712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c2f51-47b9-4a49-9a98-3a2a2aeb9926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.225356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c2f51-47b9-4a49-9a98-3a2a2aeb9926 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.225722 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c2f51-47b9-4a49-9a98-3a2a2aeb9926 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.226005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c2f51-47b9-4a49-9a98-3a2a2aeb9926 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.227013 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c2f51-47b9-4a49-9a98-3a2a2aeb9926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.228596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: WARNING py.warnings [None req-e09c2f51-47b9-4a49-9a98-3a2a2aeb9926 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 14:52:14.228596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: warnings.warn(msg) Sep 19 14:52:14.228596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: Sep 19 14:52:14.230228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c2f51-47b9-4a49-9a98-3a2a2aeb9926 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.275203 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cc9c7-8418-45a0-8ff6-4164299cda5a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.276095 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 777/1547] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:52:14.277403 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84505d-088b-4c12-95b4-24c9d1e5094b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.277690 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84505d-088b-4c12-95b4-24c9d1e5094b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.277963 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84505d-088b-4c12-95b4-24c9d1e5094b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.318663 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-6c84505d-088b-4c12-95b4-24c9d1e5094b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:14.319937 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6c84505d-088b-4c12-95b4-24c9d1e5094b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:14.398060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cbfd3-34d9-4a03-b772-bd8f8808e99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.398694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cbfd3-34d9-4a03-b772-bd8f8808e99b admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.398977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cbfd3-34d9-4a03-b772-bd8f8808e99b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.399249 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cbfd3-34d9-4a03-b772-bd8f8808e99b admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.402893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986cbfd3-34d9-4a03-b772-bd8f8808e99b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.403998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: WARNING py.warnings [None req-986cbfd3-34d9-4a03-b772-bd8f8808e99b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 14:52:14.403998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: warnings.warn(msg) Sep 19 14:52:14.403998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: Sep 19 14:52:14.408161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986cbfd3-34d9-4a03-b772-bd8f8808e99b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.413207 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 778/1548] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:14.414054 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84505d-088b-4c12-95b4-24c9d1e5094b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.430665 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84505d-088b-4c12-95b4-24c9d1e5094b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.441855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd4ead-0aed-4ad8-806d-064748390ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.442605 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd4ead-0aed-4ad8-806d-064748390ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.443053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd4ead-0aed-4ad8-806d-064748390ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.444392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd4ead-0aed-4ad8-806d-064748390ee8 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.445462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd4ead-0aed-4ad8-806d-064748390ee8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.448511 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd4ead-0aed-4ad8-806d-064748390ee8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.453722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 771/1549] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/users => generated 294 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:52:14.458044 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4bea2-22f5-4be1-8615-eae077e9b14a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.458044 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4bea2-22f5-4be1-8615-eae077e9b14a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.458044 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d4bea2-22f5-4be1-8615-eae077e9b14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.481361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c0d4bea2-22f5-4be1-8615-eae077e9b14a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:14.482716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c0d4bea2-22f5-4be1-8615-eae077e9b14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:14.486764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 779/1550] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/users => generated 288 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:14.486764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7317d7-549b-4ea3-8d07-a526f3bee2cc None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.487022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7317d7-549b-4ea3-8d07-a526f3bee2cc None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.487501 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7317d7-549b-4ea3-8d07-a526f3bee2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.496997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 772/1551] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:14.498452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe44fa8-01e4-45c2-b713-149a06b73e43 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.498745 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe44fa8-01e4-45c2-b713-149a06b73e43 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.499021 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe44fa8-01e4-45c2-b713-149a06b73e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.513778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-fb7317d7-549b-4ea3-8d07-a526f3bee2cc None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:14.514980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fb7317d7-549b-4ea3-8d07-a526f3bee2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:14.522378 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8fe44fa8-01e4-45c2-b713-149a06b73e43 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:14.523153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 780/1552] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:14.523561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8fe44fa8-01e4-45c2-b713-149a06b73e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:14.524515 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5659761-4098-4f06-8dec-e74c058ac1aa None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.524798 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5659761-4098-4f06-8dec-e74c058ac1aa None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.525597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5659761-4098-4f06-8dec-e74c058ac1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.537762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 773/1553] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:14.538931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe44fa8-01e4-45c2-b713-149a06b73e43 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.545641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b5659761-4098-4f06-8dec-e74c058ac1aa None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:14.546838 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b5659761-4098-4f06-8dec-e74c058ac1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:14.549413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe44fa8-01e4-45c2-b713-149a06b73e43 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.554067 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee059ba7-7827-441b-9050-eef50129d6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.554659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee059ba7-7827-441b-9050-eef50129d6aa admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.554904 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 781/1554] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:14.555039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee059ba7-7827-441b-9050-eef50129d6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.555226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee059ba7-7827-441b-9050-eef50129d6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.555854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee059ba7-7827-441b-9050-eef50129d6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.556479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9b2c7-ab35-4074-bb4b-15b6d6e26c40 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.556757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9b2c7-ab35-4074-bb4b-15b6d6e26c40 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.557026 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9b2c7-ab35-4074-bb4b-15b6d6e26c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.557762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee059ba7-7827-441b-9050-eef50129d6aa admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.571022 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 774/1555] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:14.571022 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee059ba7-7827-441b-9050-eef50129d6aa admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.576541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-44a9b2c7-ab35-4074-bb4b-15b6d6e26c40 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:14.577729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-44a9b2c7-ab35-4074-bb4b-15b6d6e26c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:14.579015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee059ba7-7827-441b-9050-eef50129d6aa admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.583474 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7666c11-9322-45ff-b70f-2013f122d420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.584089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7666c11-9322-45ff-b70f-2013f122d420 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.584372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7666c11-9322-45ff-b70f-2013f122d420 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.584644 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7666c11-9322-45ff-b70f-2013f122d420 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.585276 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7666c11-9322-45ff-b70f-2013f122d420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.585737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 782/1556] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:14.586857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7666c11-9322-45ff-b70f-2013f122d420 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.586981 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6e545-31ed-4a4e-95aa-42b7e60b32c5 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.587143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6e545-31ed-4a4e-95aa-42b7e60b32c5 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.587966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6e545-31ed-4a4e-95aa-42b7e60b32c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.606526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 775/1557] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:14.607527 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-cce6e545-31ed-4a4e-95aa-42b7e60b32c5 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:14.607729 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a642d7-c4b6-4705-ac19-8c12518faadc None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.608168 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a642d7-c4b6-4705-ac19-8c12518faadc None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.608317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a642d7-c4b6-4705-ac19-8c12518faadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.608906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cce6e545-31ed-4a4e-95aa-42b7e60b32c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:14.619120 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 783/1558] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:14.620466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2cd6d-2445-401b-aabd-eb73b7dc18b6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.620751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2cd6d-2445-401b-aabd-eb73b7dc18b6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.621020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2cd6d-2445-401b-aabd-eb73b7dc18b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.629380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-92a642d7-c4b6-4705-ac19-8c12518faadc None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:14.630499 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-92a642d7-c4b6-4705-ac19-8c12518faadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:14.638859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 776/1559] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:14.639652 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a642d7-c4b6-4705-ac19-8c12518faadc None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.643403 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0bb2cd6d-2445-401b-aabd-eb73b7dc18b6 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:14.644945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0bb2cd6d-2445-401b-aabd-eb73b7dc18b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:14.645720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:14.651694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a642d7-c4b6-4705-ac19-8c12518faadc None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.658068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ed840-d6a3-4774-b39e-1d47a4cba066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.658646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 784/1560] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:14.658920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ed840-d6a3-4774-b39e-1d47a4cba066 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.660050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ed840-d6a3-4774-b39e-1d47a4cba066 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.660050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ed840-d6a3-4774-b39e-1d47a4cba066 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.660304 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8145e-a221-4010-867d-29cee87ed2eb None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.660631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8145e-a221-4010-867d-29cee87ed2eb None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.660907 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8145e-a221-4010-867d-29cee87ed2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.661005 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ed840-d6a3-4774-b39e-1d47a4cba066 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:52:14.661446 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ed840-d6a3-4774-b39e-1d47a4cba066 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.663281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ed840-d6a3-4774-b39e-1d47a4cba066 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.669565 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:14.677621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 777/1561] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:52:14.678618 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ed840-d6a3-4774-b39e-1d47a4cba066 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.689701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ed840-d6a3-4774-b39e-1d47a4cba066 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.692967 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ccf8145e-a221-4010-867d-29cee87ed2eb None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:14.694164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ccf8145e-a221-4010-867d-29cee87ed2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:14.694717 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411890c-0898-4366-bd1c-fcfdcee0f5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.695328 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411890c-0898-4366-bd1c-fcfdcee0f5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.695609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411890c-0898-4366-bd1c-fcfdcee0f5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.695917 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411890c-0898-4366-bd1c-fcfdcee0f5f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.696556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411890c-0898-4366-bd1c-fcfdcee0f5f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.698651 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9411890c-0898-4366-bd1c-fcfdcee0f5f5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.703076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 785/1562] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:14.704568 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2f2eb-a944-4909-8dec-d9e22c718d86 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.704702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2f2eb-a944-4909-8dec-d9e22c718d86 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.704939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2f2eb-a944-4909-8dec-d9e22c718d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.710693 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 778/1563] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:14.714225 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411890c-0898-4366-bd1c-fcfdcee0f5f5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.725816 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e5a2f2eb-a944-4909-8dec-d9e22c718d86 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:14.727058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e5a2f2eb-a944-4909-8dec-d9e22c718d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:14.728647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9411890c-0898-4366-bd1c-fcfdcee0f5f5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.733155 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b11b4-1140-47a4-b986-d8c247bc10da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.733893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b11b4-1140-47a4-b986-d8c247bc10da admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.734698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b11b4-1140-47a4-b986-d8c247bc10da admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.734968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 786/1564] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:14.735095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b11b4-1140-47a4-b986-d8c247bc10da admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.735741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b11b4-1140-47a4-b986-d8c247bc10da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.736281 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb3fa1-0853-40db-a145-e3fb47b94409 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.736557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb3fa1-0853-40db-a145-e3fb47b94409 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.736822 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb3fa1-0853-40db-a145-e3fb47b94409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.737406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b11b4-1140-47a4-b986-d8c247bc10da admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.748009 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 779/1565] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:14.749107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b11b4-1140-47a4-b986-d8c247bc10da admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.757503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c5cb3fa1-0853-40db-a145-e3fb47b94409 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:14.758692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c5cb3fa1-0853-40db-a145-e3fb47b94409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:14.758809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b11b4-1140-47a4-b986-d8c247bc10da admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.764187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5989a9-3a24-4d8d-9274-ba2238069508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.764786 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5989a9-3a24-4d8d-9274-ba2238069508 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.765067 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5989a9-3a24-4d8d-9274-ba2238069508 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.765340 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5989a9-3a24-4d8d-9274-ba2238069508 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.766117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5989a9-3a24-4d8d-9274-ba2238069508 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:52:14.766417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5989a9-3a24-4d8d-9274-ba2238069508 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.767246 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 787/1566] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:14.768071 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5989a9-3a24-4d8d-9274-ba2238069508 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.768533 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb3fa1-0853-40db-a145-e3fb47b94409 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.780689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb3fa1-0853-40db-a145-e3fb47b94409 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.781109 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 780/1567] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:52:14.781910 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5989a9-3a24-4d8d-9274-ba2238069508 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.792028 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5989a9-3a24-4d8d-9274-ba2238069508 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.792187 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb5e26-1167-4e58-b9b8-21516d719fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.792726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb5e26-1167-4e58-b9b8-21516d719fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.793000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb5e26-1167-4e58-b9b8-21516d719fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.793266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb5e26-1167-4e58-b9b8-21516d719fc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.793901 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb5e26-1167-4e58-b9b8-21516d719fc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:52:14.794319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb5e26-1167-4e58-b9b8-21516d719fc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.796133 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb5e26-1167-4e58-b9b8-21516d719fc3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.796683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f4c11-1ce1-49fb-af19-8af1642e1725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.797588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f4c11-1ce1-49fb-af19-8af1642e1725 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.797863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f4c11-1ce1-49fb-af19-8af1642e1725 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.798136 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f4c11-1ce1-49fb-af19-8af1642e1725 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.798747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f4c11-1ce1-49fb-af19-8af1642e1725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.800667 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f4c11-1ce1-49fb-af19-8af1642e1725 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.808840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 788/1568] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:52:14.809627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb5e26-1167-4e58-b9b8-21516d719fc3 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.821447 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb5e26-1167-4e58-b9b8-21516d719fc3 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.826704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319cfb8-bda9-4a26-895a-87dbab1e8f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.827340 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319cfb8-bda9-4a26-895a-87dbab1e8f7d admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.827600 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319cfb8-bda9-4a26-895a-87dbab1e8f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.827926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319cfb8-bda9-4a26-895a-87dbab1e8f7d admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.828565 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319cfb8-bda9-4a26-895a-87dbab1e8f7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:52:14.828868 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319cfb8-bda9-4a26-895a-87dbab1e8f7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.833095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 781/1569] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:14.833263 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319cfb8-bda9-4a26-895a-87dbab1e8f7d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.833352 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f4c11-1ce1-49fb-af19-8af1642e1725 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.843370 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 789/1570] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:52:14.844221 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319cfb8-bda9-4a26-895a-87dbab1e8f7d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.854387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319cfb8-bda9-4a26-895a-87dbab1e8f7d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.908568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f4c11-1ce1-49fb-af19-8af1642e1725 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.912753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca59f68-52c6-480f-bf78-c62cfa84dc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.913115 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c59d59-48e5-4d95-9d46-bfefb31c7aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.913365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca59f68-52c6-480f-bf78-c62cfa84dc0f admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.913644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca59f68-52c6-480f-bf78-c62cfa84dc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.913772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c59d59-48e5-4d95-9d46-bfefb31c7aa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.913913 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca59f68-52c6-480f-bf78-c62cfa84dc0f admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.914053 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c59d59-48e5-4d95-9d46-bfefb31c7aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.914326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c59d59-48e5-4d95-9d46-bfefb31c7aa8 admin admin] PATH_INFO: `/v3/projects/db03071b1326441bb9571432be926bf5/users/b2c40600d2fd4eea8e4ba999050abd73/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.914542 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca59f68-52c6-480f-bf78-c62cfa84dc0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:52:14.914838 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca59f68-52c6-480f-bf78-c62cfa84dc0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.916806 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca59f68-52c6-480f-bf78-c62cfa84dc0f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.929241 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 790/1571] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:52:14.929919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca59f68-52c6-480f-bf78-c62cfa84dc0f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.930310 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c59d59-48e5-4d95-9d46-bfefb31c7aa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db03071b1326441bb9571432be926bf5, user_id=b2c40600d2fd4eea8e4ba999050abd73, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.933317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c59d59-48e5-4d95-9d46-bfefb31c7aa8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.940494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca59f68-52c6-480f-bf78-c62cfa84dc0f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:14.945775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa937b0-5624-4f0b-89df-4382a597aa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:14.946422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa937b0-5624-4f0b-89df-4382a597aa9f admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:14.946691 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa937b0-5624-4f0b-89df-4382a597aa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:14.947480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa937b0-5624-4f0b-89df-4382a597aa9f admin admin] PATH_INFO: `/v3/projects/e185bf96475e47d3a0ec27f8185ddbe9/users/0517c431d8f6424289aab070af2acee6/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:14.949866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 782/1572] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:14 2022] PUT /identity/v3/projects/db03071b1326441bb9571432be926bf5/users/b2c40600d2fd4eea8e4ba999050abd73/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:14.951685 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c59d59-48e5-4d95-9d46-bfefb31c7aa8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:14.963266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa937b0-5624-4f0b-89df-4382a597aa9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e185bf96475e47d3a0ec27f8185ddbe9, user_id=0517c431d8f6424289aab070af2acee6, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:14.964325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: WARNING py.warnings [None req-6aa937b0-5624-4f0b-89df-4382a597aa9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 14:52:14.964325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: warnings.warn(msg) Sep 19 14:52:14.964325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: Sep 19 14:52:14.966254 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa937b0-5624-4f0b-89df-4382a597aa9f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:14.980664 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 791/1573] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:14 2022] PUT /identity/v3/projects/e185bf96475e47d3a0ec27f8185ddbe9/users/0517c431d8f6424289aab070af2acee6/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:14.981207 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa937b0-5624-4f0b-89df-4382a597aa9f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.023590 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c59d59-48e5-4d95-9d46-bfefb31c7aa8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.045633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:15.047940 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa937b0-5624-4f0b-89df-4382a597aa9f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.052669 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d485d0-d81f-41e4-b9ce-03c7260a9c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.053276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d485d0-d81f-41e4-b9ce-03c7260a9c3a admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.053552 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d485d0-d81f-41e4-b9ce-03c7260a9c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.053828 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d485d0-d81f-41e4-b9ce-03c7260a9c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.054556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d485d0-d81f-41e4-b9ce-03c7260a9c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.057429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d485d0-d81f-41e4-b9ce-03c7260a9c3a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.089367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 792/1574] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:14 2022] POST /identity/v3/projects => generated 363 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:15.089367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d485d0-d81f-41e4-b9ce-03c7260a9c3a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.091339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8c0dd-2e44-4fe2-b690-70453342a11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.092005 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8c0dd-2e44-4fe2-b690-70453342a11b admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.092288 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8c0dd-2e44-4fe2-b690-70453342a11b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.092571 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8c0dd-2e44-4fe2-b690-70453342a11b admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.093248 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8c0dd-2e44-4fe2-b690-70453342a11b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:52:15.093559 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8c0dd-2e44-4fe2-b690-70453342a11b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.095236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a8c0dd-2e44-4fe2-b690-70453342a11b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.101852 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:15.114026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 783/1575] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:52:15.114026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8c0dd-2e44-4fe2-b690-70453342a11b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.121528 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a8c0dd-2e44-4fe2-b690-70453342a11b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.160044 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d485d0-d81f-41e4-b9ce-03c7260a9c3a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.164503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283dae21-41af-437c-b075-21f735a48d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.165110 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283dae21-41af-437c-b075-21f735a48d71 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.165384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283dae21-41af-437c-b075-21f735a48d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.165752 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283dae21-41af-437c-b075-21f735a48d71 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.166369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283dae21-41af-437c-b075-21f735a48d71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.168306 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283dae21-41af-437c-b075-21f735a48d71 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.179156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0a00e-d1c9-48f9-9e59-915e01d25686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.179950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0a00e-d1c9-48f9-9e59-915e01d25686 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.180234 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0a00e-d1c9-48f9-9e59-915e01d25686 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.180511 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0a00e-d1c9-48f9-9e59-915e01d25686 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.182033 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0a00e-d1c9-48f9-9e59-915e01d25686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.184008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0a00e-d1c9-48f9-9e59-915e01d25686 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.196212 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 793/1576] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:15 2022] POST /identity/v3/projects => generated 391 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:15.200443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283dae21-41af-437c-b075-21f735a48d71 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.213428 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 784/1577] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:15 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:15.213428 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0a00e-d1c9-48f9-9e59-915e01d25686 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.221543 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0a00e-d1c9-48f9-9e59-915e01d25686 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.263188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283dae21-41af-437c-b075-21f735a48d71 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.268495 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1325a51f-e650-4443-9101-a4b76ddad0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.269122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325a51f-e650-4443-9101-a4b76ddad0a7 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.269396 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325a51f-e650-4443-9101-a4b76ddad0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.269670 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1325a51f-e650-4443-9101-a4b76ddad0a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.270297 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325a51f-e650-4443-9101-a4b76ddad0a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.272272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1325a51f-e650-4443-9101-a4b76ddad0a7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.278860 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01864382-4312-42b9-8ada-9784dc3edf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.279478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01864382-4312-42b9-8ada-9784dc3edf67 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.279762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01864382-4312-42b9-8ada-9784dc3edf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.280066 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01864382-4312-42b9-8ada-9784dc3edf67 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.280810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01864382-4312-42b9-8ada-9784dc3edf67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.282759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01864382-4312-42b9-8ada-9784dc3edf67 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.300625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 794/1578] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:15 2022] POST /identity/v3/projects => generated 405 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:15.303960 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e730a-7f98-4ca2-bab8-7e5108270d0d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.304286 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e730a-7f98-4ca2-bab8-7e5108270d0d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.304704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e730a-7f98-4ca2-bab8-7e5108270d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.315130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 785/1579] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:15 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:15.315130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39800da3-d651-4dab-939d-7d977eb4b81f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.315130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39800da3-d651-4dab-939d-7d977eb4b81f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.315130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39800da3-d651-4dab-939d-7d977eb4b81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.328121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ad4e730a-7f98-4ca2-bab8-7e5108270d0d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:15.329336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ad4e730a-7f98-4ca2-bab8-7e5108270d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:15.343416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-39800da3-d651-4dab-939d-7d977eb4b81f None None] MFA Rules not processed for user `0517c431d8f6424289aab070af2acee6`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:15.344662 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-39800da3-d651-4dab-939d-7d977eb4b81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:15.365086 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:15.393221 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 795/1580] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:15.394052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e730a-7f98-4ca2-bab8-7e5108270d0d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.403177 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e730a-7f98-4ca2-bab8-7e5108270d0d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.407869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09204c8-03f8-4a17-a9cd-93b1ca23e4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.408476 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09204c8-03f8-4a17-a9cd-93b1ca23e4a3 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.408753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09204c8-03f8-4a17-a9cd-93b1ca23e4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.409020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09204c8-03f8-4a17-a9cd-93b1ca23e4a3 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.409653 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09204c8-03f8-4a17-a9cd-93b1ca23e4a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.411718 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09204c8-03f8-4a17-a9cd-93b1ca23e4a3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.414778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 786/1581] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:15.416232 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39800da3-d651-4dab-939d-7d977eb4b81f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.426627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39800da3-d651-4dab-939d-7d977eb4b81f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.432607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d4e72-e433-464a-87b8-74fa05112e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.433219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d4e72-e433-464a-87b8-74fa05112e63 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.433496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d4e72-e433-464a-87b8-74fa05112e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.433764 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d4e72-e433-464a-87b8-74fa05112e63 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.434417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d4e72-e433-464a-87b8-74fa05112e63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.435517 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 796/1582] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:15 2022] POST /identity/v3/users => generated 282 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:15.436382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d4e72-e433-464a-87b8-74fa05112e63 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.439457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09204c8-03f8-4a17-a9cd-93b1ca23e4a3 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.445501 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:15.449733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09204c8-03f8-4a17-a9cd-93b1ca23e4a3 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.455709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb2034-2603-4d18-956f-c292146f2e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.456344 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb2034-2603-4d18-956f-c292146f2e3e admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.456612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb2034-2603-4d18-956f-c292146f2e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.456898 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb2034-2603-4d18-956f-c292146f2e3e admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.457516 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb2034-2603-4d18-956f-c292146f2e3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.459550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb2034-2603-4d18-956f-c292146f2e3e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.464296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 787/1583] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:15 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:15.468317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d4e72-e433-464a-87b8-74fa05112e63 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.483620 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 797/1584] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:15 2022] POST /identity/v3/users => generated 296 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:15.484681 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb2034-2603-4d18-956f-c292146f2e3e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.494802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb2034-2603-4d18-956f-c292146f2e3e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.539649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d4e72-e433-464a-87b8-74fa05112e63 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.544927 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bb474-c158-414a-94b1-1919f7d67537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.545668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bb474-c158-414a-94b1-1919f7d67537 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.545945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bb474-c158-414a-94b1-1919f7d67537 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.546223 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547bb474-c158-414a-94b1-1919f7d67537 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.546859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bb474-c158-414a-94b1-1919f7d67537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.548520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547bb474-c158-414a-94b1-1919f7d67537 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.559439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 788/1585] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:15.560270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bb474-c158-414a-94b1-1919f7d67537 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.561953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8097b-4268-420f-b5f4-f4f824c7f865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.562580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8097b-4268-420f-b5f4-f4f824c7f865 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.562883 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8097b-4268-420f-b5f4-f4f824c7f865 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.563268 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e8097b-4268-420f-b5f4-f4f824c7f865 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.563786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e8097b-4268-420f-b5f4-f4f824c7f865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.565742 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e8097b-4268-420f-b5f4-f4f824c7f865 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.572584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547bb474-c158-414a-94b1-1919f7d67537 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.577529 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68129211-245a-4586-9449-da6d1e782e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.578349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68129211-245a-4586-9449-da6d1e782e1d admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.578606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68129211-245a-4586-9449-da6d1e782e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.578962 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68129211-245a-4586-9449-da6d1e782e1d admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.579678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68129211-245a-4586-9449-da6d1e782e1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.581610 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68129211-245a-4586-9449-da6d1e782e1d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.588005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 798/1586] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:15 2022] POST /identity/v3/users => generated 303 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:15.588840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8097b-4268-420f-b5f4-f4f824c7f865 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.604111 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 789/1587] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:15 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:15.605002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68129211-245a-4586-9449-da6d1e782e1d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.614865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68129211-245a-4586-9449-da6d1e782e1d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.620168 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f6220-9f8d-4e6d-b1d3-a59fa4639f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.620785 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f6220-9f8d-4e6d-b1d3-a59fa4639f8a admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.621073 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f6220-9f8d-4e6d-b1d3-a59fa4639f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.621396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f6220-9f8d-4e6d-b1d3-a59fa4639f8a admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.622177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f6220-9f8d-4e6d-b1d3-a59fa4639f8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.624303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f6220-9f8d-4e6d-b1d3-a59fa4639f8a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.646328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e8097b-4268-420f-b5f4-f4f824c7f865 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.650009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426955e-9d95-4538-b271-f88121ce8182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.650620 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426955e-9d95-4538-b271-f88121ce8182 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.650900 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426955e-9d95-4538-b271-f88121ce8182 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.651171 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426955e-9d95-4538-b271-f88121ce8182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.653128 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 790/1588] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:15 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:15.654256 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f6220-9f8d-4e6d-b1d3-a59fa4639f8a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.663928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8426955e-9d95-4538-b271-f88121ce8182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.665307 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8426955e-9d95-4538-b271-f88121ce8182 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.676757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 799/1589] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:15.677574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426955e-9d95-4538-b271-f88121ce8182 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.688566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426955e-9d95-4538-b271-f88121ce8182 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.721219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:15.727740 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f6220-9f8d-4e6d-b1d3-a59fa4639f8a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.733918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fee330-eba7-4191-9075-d641fb412ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.734652 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fee330-eba7-4191-9075-d641fb412ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.734990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fee330-eba7-4191-9075-d641fb412ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.735320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fee330-eba7-4191-9075-d641fb412ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.736048 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fee330-eba7-4191-9075-d641fb412ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.737734 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fee330-eba7-4191-9075-d641fb412ef2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.751141 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 791/1590] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:15.752107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fee330-eba7-4191-9075-d641fb412ef2 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.762476 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fee330-eba7-4191-9075-d641fb412ef2 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.762717 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f02d6f-6bcd-4369-a9ea-45b369b4c62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.763208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f02d6f-6bcd-4369-a9ea-45b369b4c62b admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.763658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f02d6f-6bcd-4369-a9ea-45b369b4c62b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.764000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f02d6f-6bcd-4369-a9ea-45b369b4c62b admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.764802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f02d6f-6bcd-4369-a9ea-45b369b4c62b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.767119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f02d6f-6bcd-4369-a9ea-45b369b4c62b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.768368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eaea37-6484-4f54-84ab-78ffb8a97610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.769089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eaea37-6484-4f54-84ab-78ffb8a97610 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.769493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eaea37-6484-4f54-84ab-78ffb8a97610 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.769831 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eaea37-6484-4f54-84ab-78ffb8a97610 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.770466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eaea37-6484-4f54-84ab-78ffb8a97610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.772162 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eaea37-6484-4f54-84ab-78ffb8a97610 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.784946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 792/1591] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:15.786081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eaea37-6484-4f54-84ab-78ffb8a97610 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.794985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 800/1592] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:15 2022] POST /identity/v3/users => generated 286 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:15.796118 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f02d6f-6bcd-4369-a9ea-45b369b4c62b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.801028 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eaea37-6484-4f54-84ab-78ffb8a97610 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.809916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f02d6f-6bcd-4369-a9ea-45b369b4c62b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.810143 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a078f-1d98-4c7a-a427-7ad8d40a08a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.810750 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a078f-1d98-4c7a-a427-7ad8d40a08a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.811064 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a078f-1d98-4c7a-a427-7ad8d40a08a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.811323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a078f-1d98-4c7a-a427-7ad8d40a08a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.812037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a078f-1d98-4c7a-a427-7ad8d40a08a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.813809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a078f-1d98-4c7a-a427-7ad8d40a08a9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.816540 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ceb8c-2bbe-4226-9a72-10f2256badb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.817255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ceb8c-2bbe-4226-9a72-10f2256badb6 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.817658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ceb8c-2bbe-4226-9a72-10f2256badb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.818151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ceb8c-2bbe-4226-9a72-10f2256badb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.819118 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ceb8c-2bbe-4226-9a72-10f2256badb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.821262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ceb8c-2bbe-4226-9a72-10f2256badb6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.824803 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:15.824997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 793/1593] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:15.829018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a078f-1d98-4c7a-a427-7ad8d40a08a9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.836824 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 801/1594] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:15.838966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ceb8c-2bbe-4226-9a72-10f2256badb6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.843185 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a078f-1d98-4c7a-a427-7ad8d40a08a9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.848992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57542246-d4ff-4b27-a8fa-d89b275f7d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.849627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57542246-d4ff-4b27-a8fa-d89b275f7d38 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.849899 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57542246-d4ff-4b27-a8fa-d89b275f7d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.850172 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57542246-d4ff-4b27-a8fa-d89b275f7d38 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.850891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57542246-d4ff-4b27-a8fa-d89b275f7d38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.851404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ceb8c-2bbe-4226-9a72-10f2256badb6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.852561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57542246-d4ff-4b27-a8fa-d89b275f7d38 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.857548 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbdb379-590c-4af2-8ea4-bcaae63b2d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.858297 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdb379-590c-4af2-8ea4-bcaae63b2d7f admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.858652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdb379-590c-4af2-8ea4-bcaae63b2d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.859031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdb379-590c-4af2-8ea4-bcaae63b2d7f admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.859768 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbdb379-590c-4af2-8ea4-bcaae63b2d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.861833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbdb379-590c-4af2-8ea4-bcaae63b2d7f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.864277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 794/1595] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:15.864969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57542246-d4ff-4b27-a8fa-d89b275f7d38 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.878316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57542246-d4ff-4b27-a8fa-d89b275f7d38 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.896075 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3fd0a-c332-4caa-9a18-ef4e7f58900b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.896915 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3fd0a-c332-4caa-9a18-ef4e7f58900b admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.897198 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3fd0a-c332-4caa-9a18-ef4e7f58900b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.897470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3fd0a-c332-4caa-9a18-ef4e7f58900b admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.898643 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3fd0a-c332-4caa-9a18-ef4e7f58900b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.902413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3fd0a-c332-4caa-9a18-ef4e7f58900b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.911131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 802/1596] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:15 2022] POST /identity/v3/users => generated 294 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:15.912008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbdb379-590c-4af2-8ea4-bcaae63b2d7f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.919015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 795/1597] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:15.919015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3fd0a-c332-4caa-9a18-ef4e7f58900b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.926900 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3fd0a-c332-4caa-9a18-ef4e7f58900b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.927544 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbdb379-590c-4af2-8ea4-bcaae63b2d7f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.932949 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7ec77-a965-4f88-9bf2-7193378ddf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.933574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7ec77-a965-4f88-9bf2-7193378ddf0a admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.933849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7ec77-a965-4f88-9bf2-7193378ddf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.934124 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7ec77-a965-4f88-9bf2-7193378ddf0a admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.934795 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7ec77-a965-4f88-9bf2-7193378ddf0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.936427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e7ec77-a965-4f88-9bf2-7193378ddf0a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.937289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650030b5-8082-45e0-a3de-33d6e712d331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.938165 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650030b5-8082-45e0-a3de-33d6e712d331 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.939149 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650030b5-8082-45e0-a3de-33d6e712d331 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.939686 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650030b5-8082-45e0-a3de-33d6e712d331 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.940937 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650030b5-8082-45e0-a3de-33d6e712d331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.944706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650030b5-8082-45e0-a3de-33d6e712d331 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.948238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 796/1598] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:15.952210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7ec77-a965-4f88-9bf2-7193378ddf0a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.960753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 803/1599] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:15.960753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650030b5-8082-45e0-a3de-33d6e712d331 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.964499 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7ec77-a965-4f88-9bf2-7193378ddf0a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.970194 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e463e-ea95-474c-a9ee-7564f0527a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.971009 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e463e-ea95-474c-a9ee-7564f0527a14 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.971392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e463e-ea95-474c-a9ee-7564f0527a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.971770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e463e-ea95-474c-a9ee-7564f0527a14 admin admin] PATH_INFO: `/v3/projects/a0009b7d4f2545ffab964938384d40b6/users/49b775647ffb4500911c8ec8cf2ec804/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.971912 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650030b5-8082-45e0-a3de-33d6e712d331 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:15.976833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ce2c6-09a1-41b1-9ebf-cc10da0fcac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:15.977456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ce2c6-09a1-41b1-9ebf-cc10da0fcac2 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:15.977738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ce2c6-09a1-41b1-9ebf-cc10da0fcac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:15.978032 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ce2c6-09a1-41b1-9ebf-cc10da0fcac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:15.979295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ce2c6-09a1-41b1-9ebf-cc10da0fcac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.980861 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ce2c6-09a1-41b1-9ebf-cc10da0fcac2 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:15.991485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e463e-ea95-474c-a9ee-7564f0527a14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0009b7d4f2545ffab964938384d40b6, user_id=49b775647ffb4500911c8ec8cf2ec804, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:15.994337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 804/1600] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:15.995325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ce2c6-09a1-41b1-9ebf-cc10da0fcac2 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:15.998008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e463e-ea95-474c-a9ee-7564f0527a14 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:16.008961 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ce2c6-09a1-41b1-9ebf-cc10da0fcac2 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:16.014515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 797/1601] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:15 2022] PUT /identity/v3/projects/a0009b7d4f2545ffab964938384d40b6/users/49b775647ffb4500911c8ec8cf2ec804/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:16.018809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e463e-ea95-474c-a9ee-7564f0527a14 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:16.085101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bce1e-3a8b-4ed2-8245-4fde9f0d7e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:16.085761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bce1e-3a8b-4ed2-8245-4fde9f0d7e53 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.086075 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bce1e-3a8b-4ed2-8245-4fde9f0d7e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.086399 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135bce1e-3a8b-4ed2-8245-4fde9f0d7e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.087030 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bce1e-3a8b-4ed2-8245-4fde9f0d7e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:16.088825 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135bce1e-3a8b-4ed2-8245-4fde9f0d7e53 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:16.105203 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 805/1602] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:16.107002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bce1e-3a8b-4ed2-8245-4fde9f0d7e53 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:16.117833 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e463e-ea95-474c-a9ee-7564f0527a14 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:16.121620 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135bce1e-3a8b-4ed2-8245-4fde9f0d7e53 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:16.130540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43857118-5831-4d1c-8cd5-b51a7b6ee8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:16.131302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43857118-5831-4d1c-8cd5-b51a7b6ee8ed admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.131413 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d46b7-47f6-42f8-8f11-1d98b6b4e022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:16.131636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43857118-5831-4d1c-8cd5-b51a7b6ee8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.131883 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43857118-5831-4d1c-8cd5-b51a7b6ee8ed admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.132012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d46b7-47f6-42f8-8f11-1d98b6b4e022 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.132274 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d46b7-47f6-42f8-8f11-1d98b6b4e022 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.132519 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43857118-5831-4d1c-8cd5-b51a7b6ee8ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:16.132631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d46b7-47f6-42f8-8f11-1d98b6b4e022 admin admin] PATH_INFO: `/v3/projects/e8d827ccb2fe4b3e81e4eef92a0035c5/users/9a8f128e3bd8499abea2a71532eb4cf6/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.134171 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43857118-5831-4d1c-8cd5-b51a7b6ee8ed admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:16.145595 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 798/1603] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:16.149356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43857118-5831-4d1c-8cd5-b51a7b6ee8ed admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:16.151631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d46b7-47f6-42f8-8f11-1d98b6b4e022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8d827ccb2fe4b3e81e4eef92a0035c5, user_id=9a8f128e3bd8499abea2a71532eb4cf6, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:16.153911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d46b7-47f6-42f8-8f11-1d98b6b4e022 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:16.160810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43857118-5831-4d1c-8cd5-b51a7b6ee8ed admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:16.166233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a0043-da60-4106-a085-4a29fe435746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:16.166846 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a0043-da60-4106-a085-4a29fe435746 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.167295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a0043-da60-4106-a085-4a29fe435746 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.167633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a0043-da60-4106-a085-4a29fe435746 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.168244 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a0043-da60-4106-a085-4a29fe435746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:16.169982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a0043-da60-4106-a085-4a29fe435746 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:16.175679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 806/1604] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:16 2022] PUT /identity/v3/projects/e8d827ccb2fe4b3e81e4eef92a0035c5/users/9a8f128e3bd8499abea2a71532eb4cf6/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:16.176694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d46b7-47f6-42f8-8f11-1d98b6b4e022 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:16.185728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 799/1605] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:16.185728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a0043-da60-4106-a085-4a29fe435746 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:16.199697 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a0043-da60-4106-a085-4a29fe435746 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:16.260409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d46b7-47f6-42f8-8f11-1d98b6b4e022 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:16.265456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb11a8a-b460-4187-a0f9-54d79ff00fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:16.266137 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb11a8a-b460-4187-a0f9-54d79ff00fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.266399 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb11a8a-b460-4187-a0f9-54d79ff00fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.266671 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb11a8a-b460-4187-a0f9-54d79ff00fb0 admin admin] PATH_INFO: `/v3/projects/73c223bd20fc43eca4c1cd0921f7906d/users/1cccf354894c4311b7710d8a65682eeb/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.283651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb11a8a-b460-4187-a0f9-54d79ff00fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c223bd20fc43eca4c1cd0921f7906d, user_id=1cccf354894c4311b7710d8a65682eeb, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:16.286456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb11a8a-b460-4187-a0f9-54d79ff00fb0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:16.286675 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a6926-d85a-42a4-8117-69d556b1feea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:16.287349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a6926-d85a-42a4-8117-69d556b1feea admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.287630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a6926-d85a-42a4-8117-69d556b1feea admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.287998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a6926-d85a-42a4-8117-69d556b1feea admin admin] PATH_INFO: `/v3/projects/884c136dd0b443f4a0aa1822bf808aab/users/ab63f77cf82541feac9f238c301a797a/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.292200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:16.301982 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 807/1606] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:16 2022] PUT /identity/v3/projects/73c223bd20fc43eca4c1cd0921f7906d/users/1cccf354894c4311b7710d8a65682eeb/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:16.307252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0291ac-f6ba-4900-bd6f-d864639383dd None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.307540 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0291ac-f6ba-4900-bd6f-d864639383dd None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.307816 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0291ac-f6ba-4900-bd6f-d864639383dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.313044 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a6926-d85a-42a4-8117-69d556b1feea admin admin] RBAC: Authorizing `identity:create_grant(project_id=884c136dd0b443f4a0aa1822bf808aab, user_id=ab63f77cf82541feac9f238c301a797a, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:16.316355 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a6926-d85a-42a4-8117-69d556b1feea admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:16.327511 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9e0291ac-f6ba-4900-bd6f-d864639383dd None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:16.328888 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9e0291ac-f6ba-4900-bd6f-d864639383dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:16.333319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 800/1607] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:16 2022] PUT /identity/v3/projects/884c136dd0b443f4a0aa1822bf808aab/users/ab63f77cf82541feac9f238c301a797a/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:16.334141 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a6926-d85a-42a4-8117-69d556b1feea admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:16.341798 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:16.405915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 808/1608] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:16.406566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0291ac-f6ba-4900-bd6f-d864639383dd None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:16.417443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0291ac-f6ba-4900-bd6f-d864639383dd None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:16.422839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0493376-51b5-4091-874a-c3a0f7a72fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:16.423637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0493376-51b5-4091-874a-c3a0f7a72fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.424032 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0493376-51b5-4091-874a-c3a0f7a72fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.424417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0493376-51b5-4091-874a-c3a0f7a72fcf admin admin] PATH_INFO: `/v3/projects/a79d80ad80824c2d8ecb9087eb03bd66/users/6a661ca101dd415799678eefd27aceac/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.432104 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a6926-d85a-42a4-8117-69d556b1feea admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:16.439099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae253e5f-736b-4eca-98ba-cabfaec1194a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:16.440261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae253e5f-736b-4eca-98ba-cabfaec1194a admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.440728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae253e5f-736b-4eca-98ba-cabfaec1194a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.441205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae253e5f-736b-4eca-98ba-cabfaec1194a admin admin] PATH_INFO: `/v3/projects/b89056beeba940128ec6ff72b0c55b34/users/61743a63def7451d9b739f3b17afe8b8/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.449117 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0493376-51b5-4091-874a-c3a0f7a72fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a79d80ad80824c2d8ecb9087eb03bd66, user_id=6a661ca101dd415799678eefd27aceac, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:16.452266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0493376-51b5-4091-874a-c3a0f7a72fcf admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:16.461811 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae253e5f-736b-4eca-98ba-cabfaec1194a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b89056beeba940128ec6ff72b0c55b34, user_id=61743a63def7451d9b739f3b17afe8b8, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:16.465656 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae253e5f-736b-4eca-98ba-cabfaec1194a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:16.471036 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 809/1609] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:16 2022] PUT /identity/v3/projects/a79d80ad80824c2d8ecb9087eb03bd66/users/6a661ca101dd415799678eefd27aceac/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:16.471860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0493376-51b5-4091-874a-c3a0f7a72fcf admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:16.486666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 801/1610] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:16 2022] PUT /identity/v3/projects/b89056beeba940128ec6ff72b0c55b34/users/61743a63def7451d9b739f3b17afe8b8/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:16.487915 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252d51c-ae07-4687-988a-56557af1ad3b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.488800 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252d51c-ae07-4687-988a-56557af1ad3b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.488800 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252d51c-ae07-4687-988a-56557af1ad3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.546181 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2252d51c-ae07-4687-988a-56557af1ad3b None None] MFA Rules not processed for user `9a8f128e3bd8499abea2a71532eb4cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:16.548425 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2252d51c-ae07-4687-988a-56557af1ad3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:16.551084 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0493376-51b5-4091-874a-c3a0f7a72fcf admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:16.625437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1c12d-c474-4305-9ca5-17965e4fe01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:16.626238 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1c12d-c474-4305-9ca5-17965e4fe01d admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.626527 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1c12d-c474-4305-9ca5-17965e4fe01d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.626811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1c12d-c474-4305-9ca5-17965e4fe01d admin admin] PATH_INFO: `/v3/projects/505a2b7bf4e64b689bd8081324cb729e/users/97342c4d7ffc40a29286e2457dc06615/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.641513 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 802/1611] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:16.643493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32adce4-1b52-44c9-92f0-62dba8ed6906 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.644014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32adce4-1b52-44c9-92f0-62dba8ed6906 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.644511 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32adce4-1b52-44c9-92f0-62dba8ed6906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.648978 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1c12d-c474-4305-9ca5-17965e4fe01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=505a2b7bf4e64b689bd8081324cb729e, user_id=97342c4d7ffc40a29286e2457dc06615, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:16.652453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1c12d-c474-4305-9ca5-17965e4fe01d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:16.669243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d32adce4-1b52-44c9-92f0-62dba8ed6906 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:16.670879 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d32adce4-1b52-44c9-92f0-62dba8ed6906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:16.671932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 810/1612] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:16 2022] PUT /identity/v3/projects/505a2b7bf4e64b689bd8081324cb729e/users/97342c4d7ffc40a29286e2457dc06615/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:16.673631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8d104-f6e7-4d69-b467-fa5508412177 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.673950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8d104-f6e7-4d69-b467-fa5508412177 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.674204 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8d104-f6e7-4d69-b467-fa5508412177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.712965 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-47c8d104-f6e7-4d69-b467-fa5508412177 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:16.714809 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-47c8d104-f6e7-4d69-b467-fa5508412177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:16.732926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:16.745057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:16.763855 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 803/1613] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:16.765145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32adce4-1b52-44c9-92f0-62dba8ed6906 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:16.802306 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 811/1614] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:16.802858 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8d104-f6e7-4d69-b467-fa5508412177 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:16.815272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8d104-f6e7-4d69-b467-fa5508412177 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:16.822984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0307cf-ffee-403c-841a-e8c31cc4e325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:16.824639 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0307cf-ffee-403c-841a-e8c31cc4e325 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.825009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0307cf-ffee-403c-841a-e8c31cc4e325 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.825357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0307cf-ffee-403c-841a-e8c31cc4e325 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.826196 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0307cf-ffee-403c-841a-e8c31cc4e325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:16.828133 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0307cf-ffee-403c-841a-e8c31cc4e325 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:16.848192 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 812/1615] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:16.848192 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4624b1-165f-4283-86b9-4e43fa0c5c07 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.848918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4624b1-165f-4283-86b9-4e43fa0c5c07 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.849171 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4624b1-165f-4283-86b9-4e43fa0c5c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.856384 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32adce4-1b52-44c9-92f0-62dba8ed6906 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:16.861939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c80d5-830d-4d53-a9c3-fee869f21dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:16.863049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c80d5-830d-4d53-a9c3-fee869f21dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.863188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c80d5-830d-4d53-a9c3-fee869f21dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.863381 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c80d5-830d-4d53-a9c3-fee869f21dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.873672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ca4624b1-165f-4283-86b9-4e43fa0c5c07 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:16.874416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c80d5-830d-4d53-a9c3-fee869f21dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:16.875591 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ca4624b1-165f-4283-86b9-4e43fa0c5c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:16.876026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c80d5-830d-4d53-a9c3-fee869f21dd1 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:16.888699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 813/1616] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:16.889788 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4624b1-165f-4283-86b9-4e43fa0c5c07 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:16.898708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 804/1617] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:16.900481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f96d12-7eef-42c0-b413-172480603776 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.900627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4624b1-165f-4283-86b9-4e43fa0c5c07 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:16.902624 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f96d12-7eef-42c0-b413-172480603776 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.903607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f96d12-7eef-42c0-b413-172480603776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.911371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb28998-4673-40e3-8dfe-b58c89342db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:16.912202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb28998-4673-40e3-8dfe-b58c89342db6 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:16.912552 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb28998-4673-40e3-8dfe-b58c89342db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:16.912949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb28998-4673-40e3-8dfe-b58c89342db6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:16.913678 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb28998-4673-40e3-8dfe-b58c89342db6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:16.916242 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb28998-4673-40e3-8dfe-b58c89342db6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:16.957069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 814/1618] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:16 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:16.957988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb28998-4673-40e3-8dfe-b58c89342db6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:16.970969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-70f96d12-7eef-42c0-b413-172480603776 None None] MFA Rules not processed for user `97342c4d7ffc40a29286e2457dc06615`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:16.975904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-70f96d12-7eef-42c0-b413-172480603776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:17.108345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb28998-4673-40e3-8dfe-b58c89342db6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:17.116504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71838045-d212-450b-8344-c431b4098fed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:17.117152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71838045-d212-450b-8344-c431b4098fed service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:17.117437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71838045-d212-450b-8344-c431b4098fed service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:17.117716 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71838045-d212-450b-8344-c431b4098fed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:17.150606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 805/1619] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:17.152259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f96d12-7eef-42c0-b413-172480603776 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:17.163434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f96d12-7eef-42c0-b413-172480603776 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:17.169366 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57f607-5200-4a47-bd50-8414df1c2f62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:17.170004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57f607-5200-4a47-bd50-8414df1c2f62 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:17.170291 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57f607-5200-4a47-bd50-8414df1c2f62 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:17.170624 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57f607-5200-4a47-bd50-8414df1c2f62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:17.238528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71838045-d212-450b-8344-c431b4098fed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:17.240481 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71838045-d212-450b-8344-c431b4098fed service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:17.256600 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 815/1620] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:17.257607 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71838045-d212-450b-8344-c431b4098fed service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:17.269220 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71838045-d212-450b-8344-c431b4098fed service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:17.275988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2561ad8-4c48-4289-babe-edb21439d1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:17.276825 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2561ad8-4c48-4289-babe-edb21439d1ae admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:17.277884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2561ad8-4c48-4289-babe-edb21439d1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:17.278271 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2561ad8-4c48-4289-babe-edb21439d1ae admin admin] PATH_INFO: `/v3/projects/a79d80ad80824c2d8ecb9087eb03bd66/users/6a661ca101dd415799678eefd27aceac/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:17.279715 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57f607-5200-4a47-bd50-8414df1c2f62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:17.280949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2561ad8-4c48-4289-babe-edb21439d1ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=a79d80ad80824c2d8ecb9087eb03bd66, user_id=6a661ca101dd415799678eefd27aceac, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:17.281854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57f607-5200-4a47-bd50-8414df1c2f62 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:17.284938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2561ad8-4c48-4289-babe-edb21439d1ae admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:17.304322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 816/1621] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:17 2022] PUT /identity/v3/projects/a79d80ad80824c2d8ecb9087eb03bd66/users/6a661ca101dd415799678eefd27aceac/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:17.307764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2561ad8-4c48-4289-babe-edb21439d1ae admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:17.310432 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 806/1622] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:17.314519 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57f607-5200-4a47-bd50-8414df1c2f62 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:17.345671 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57f607-5200-4a47-bd50-8414df1c2f62 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:17.369939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:17.393021 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:17.456059 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2561ad8-4c48-4289-babe-edb21439d1ae admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:17.471799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cb250-92d4-466e-bb87-354f4704be30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:17.472502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cb250-92d4-466e-bb87-354f4704be30 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:17.472785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cb250-92d4-466e-bb87-354f4704be30 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:17.473061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cb250-92d4-466e-bb87-354f4704be30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:17.473184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5a90b-b502-4e75-8640-f7b230b1fc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:17.473883 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5a90b-b502-4e75-8640-f7b230b1fc3c admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:17.474178 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5a90b-b502-4e75-8640-f7b230b1fc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:17.474510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5a90b-b502-4e75-8640-f7b230b1fc3c admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:17.475246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5a90b-b502-4e75-8640-f7b230b1fc3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:17.477682 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5a90b-b502-4e75-8640-f7b230b1fc3c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:17.489761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cb250-92d4-466e-bb87-354f4704be30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:17.494593 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cb250-92d4-466e-bb87-354f4704be30 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:17.509304 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 817/1623] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:17.511128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0b035-038a-4829-883d-72c45ef64b4b None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:17.511412 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0b035-038a-4829-883d-72c45ef64b4b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:17.511738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0b035-038a-4829-883d-72c45ef64b4b None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:17.514317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 818/1624] 10.222.0.60 () {54 vars in 965 bytes} [Mon Sep 19 14:52:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:52:17.515857 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4216841-ec0d-4f81-a7d6-d1795a5499c5 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:17.516155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4216841-ec0d-4f81-a7d6-d1795a5499c5 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:17.516474 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4216841-ec0d-4f81-a7d6-d1795a5499c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:17.532406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 807/1625] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:17 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:17.532406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c8ac6-cde7-4a86-8911-6cfd35ddc977 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:17.532406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c8ac6-cde7-4a86-8911-6cfd35ddc977 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:17.532406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0c8ac6-cde7-4a86-8911-6cfd35ddc977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:17.557353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d4216841-ec0d-4f81-a7d6-d1795a5499c5 None None] MFA Rules not processed for user `6a661ca101dd415799678eefd27aceac`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:17.559660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d4216841-ec0d-4f81-a7d6-d1795a5499c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:17.596952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8c0c8ac6-cde7-4a86-8911-6cfd35ddc977 None None] MFA Rules not processed for user `59b5d2fd362b417c9b9be75f88a667ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:17.598903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8c0c8ac6-cde7-4a86-8911-6cfd35ddc977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:17.731562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 819/1626] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:17.734668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4216841-ec0d-4f81-a7d6-d1795a5499c5 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:17.759763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4216841-ec0d-4f81-a7d6-d1795a5499c5 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:17.769666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 808/1627] 10.222.0.60 () {58 vars in 1062 bytes} [Mon Sep 19 14:52:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:17.770375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1112817-4817-42be-b543-d4571159403c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:17.770804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c8ac6-cde7-4a86-8911-6cfd35ddc977 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:17.771733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1112817-4817-42be-b543-d4571159403c admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:17.772320 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1112817-4817-42be-b543-d4571159403c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:17.772771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1112817-4817-42be-b543-d4571159403c admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:17.773821 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1112817-4817-42be-b543-d4571159403c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:17.776987 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1112817-4817-42be-b543-d4571159403c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:17.792729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 820/1628] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:17.794932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1112817-4817-42be-b543-d4571159403c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:17.814568 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1112817-4817-42be-b543-d4571159403c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:17.828916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d0d37-c12f-4897-b6bc-1d73e03326b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:17.829684 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d0d37-c12f-4897-b6bc-1d73e03326b2 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:17.830015 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d0d37-c12f-4897-b6bc-1d73e03326b2 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:17.830334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d0d37-c12f-4897-b6bc-1d73e03326b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:17.874853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0c8ac6-cde7-4a86-8911-6cfd35ddc977 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:17.880679 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c5e23-9cfc-471d-8bbe-d1893fd62ecc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:17.881156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c5e23-9cfc-471d-8bbe-d1893fd62ecc service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:17.881752 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c5e23-9cfc-471d-8bbe-d1893fd62ecc service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:17.883645 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c5e23-9cfc-471d-8bbe-d1893fd62ecc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:17.972175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d0d37-c12f-4897-b6bc-1d73e03326b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:17.975641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d0d37-c12f-4897-b6bc-1d73e03326b2 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:17.982048 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:18.003951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 821/1629] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:18.005187 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d0d37-c12f-4897-b6bc-1d73e03326b2 service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:18.005894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c5e23-9cfc-471d-8bbe-d1893fd62ecc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:18.007642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c5e23-9cfc-471d-8bbe-d1893fd62ecc service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:18.021553 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 809/1630] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:18.024529 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d0d37-c12f-4897-b6bc-1d73e03326b2 service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:18.031644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064f3d0-451b-4c90-aaa5-244e5ed559c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:18.032438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064f3d0-451b-4c90-aaa5-244e5ed559c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:18.032899 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064f3d0-451b-4c90-aaa5-244e5ed559c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:18.033228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064f3d0-451b-4c90-aaa5-244e5ed559c5 admin admin] PATH_INFO: `/v3/projects/bd410f7316a740d1b309a39dcf9dfe2e/users/ced4b56fdd1b48afa2eaca18ebce7c0c/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:18.068919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064f3d0-451b-4c90-aaa5-244e5ed559c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd410f7316a740d1b309a39dcf9dfe2e, user_id=ced4b56fdd1b48afa2eaca18ebce7c0c, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:18.072598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c064f3d0-451b-4c90-aaa5-244e5ed559c5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:18.101909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 822/1631] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:18 2022] PUT /identity/v3/projects/bd410f7316a740d1b309a39dcf9dfe2e/users/ced4b56fdd1b48afa2eaca18ebce7c0c/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:18.107341 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c5e23-9cfc-471d-8bbe-d1893fd62ecc service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:18.126635 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c5e23-9cfc-471d-8bbe-d1893fd62ecc service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:18.268403 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c857800-d31a-4951-9332-3cc0e5fec1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:18.270203 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c857800-d31a-4951-9332-3cc0e5fec1ec admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:18.270488 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c857800-d31a-4951-9332-3cc0e5fec1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:18.270840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c857800-d31a-4951-9332-3cc0e5fec1ec admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:18.271390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c857800-d31a-4951-9332-3cc0e5fec1ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:18.273061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c857800-d31a-4951-9332-3cc0e5fec1ec admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:18.297365 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 810/1632] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:18 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:18.298669 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064f3d0-451b-4c90-aaa5-244e5ed559c5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:18.317813 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064f3d0-451b-4c90-aaa5-244e5ed559c5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:18.345860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b012bab0-6e97-409a-a16c-76bccf29812c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:18.346976 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012bab0-6e97-409a-a16c-76bccf29812c admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:18.346976 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012bab0-6e97-409a-a16c-76bccf29812c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:18.347209 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012bab0-6e97-409a-a16c-76bccf29812c admin admin] PATH_INFO: `/v3/projects/bd410f7316a740d1b309a39dcf9dfe2e/users/ced4b56fdd1b48afa2eaca18ebce7c0c/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:18.351018 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012bab0-6e97-409a-a16c-76bccf29812c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd410f7316a740d1b309a39dcf9dfe2e, user_id=ced4b56fdd1b48afa2eaca18ebce7c0c, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:18.354580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b012bab0-6e97-409a-a16c-76bccf29812c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:18.361465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:18.395079 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 823/1633] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:18 2022] PUT /identity/v3/projects/bd410f7316a740d1b309a39dcf9dfe2e/users/ced4b56fdd1b48afa2eaca18ebce7c0c/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:18.398697 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1159f56-3abf-4d65-ab36-afd455344aea None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:18.399000 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1159f56-3abf-4d65-ab36-afd455344aea None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:18.399352 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1159f56-3abf-4d65-ab36-afd455344aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:18.497296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f1159f56-3abf-4d65-ab36-afd455344aea None None] MFA Rules not processed for user `ced4b56fdd1b48afa2eaca18ebce7c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:18.504108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f1159f56-3abf-4d65-ab36-afd455344aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:18.648170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebcf8c-b6db-4473-9220-7c72c147e9df None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:18.648486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebcf8c-b6db-4473-9220-7c72c147e9df None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:18.648486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebcf8c-b6db-4473-9220-7c72c147e9df None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:18.650551 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 824/1634] 10.222.0.60 () {54 vars in 912 bytes} [Mon Sep 19 14:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:52:18.656700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941c9f9-1f8c-4e66-a5d4-4bb274678c20 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:18.656986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941c9f9-1f8c-4e66-a5d4-4bb274678c20 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:18.657260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941c9f9-1f8c-4e66-a5d4-4bb274678c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:18.703780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 811/1635] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:18.712036 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8941c9f9-1f8c-4e66-a5d4-4bb274678c20 None None] MFA Rules not processed for user `59b5d2fd362b417c9b9be75f88a667ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:18.713838 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8941c9f9-1f8c-4e66-a5d4-4bb274678c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:18.717344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1159f56-3abf-4d65-ab36-afd455344aea None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:18.892329 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1159f56-3abf-4d65-ab36-afd455344aea None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:18.900761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b0dce-2b5c-4aa3-b3dc-de93ce9a9a17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:18.901463 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b0dce-2b5c-4aa3-b3dc-de93ce9a9a17 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:18.901948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b0dce-2b5c-4aa3-b3dc-de93ce9a9a17 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:18.902235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b0dce-2b5c-4aa3-b3dc-de93ce9a9a17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:18.910844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 825/1636] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Sep 19 14:52:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:52:18.938595 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b0dce-2b5c-4aa3-b3dc-de93ce9a9a17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:18.940118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b0dce-2b5c-4aa3-b3dc-de93ce9a9a17 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:18.972360 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 812/1637] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:52:19.025423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941c9f9-1f8c-4e66-a5d4-4bb274678c20 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:19.038282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941c9f9-1f8c-4e66-a5d4-4bb274678c20 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:19.048004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2bef5-cdcf-40c1-a94c-279051e7031d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:19.048704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2bef5-cdcf-40c1-a94c-279051e7031d service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:19.049069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2bef5-cdcf-40c1-a94c-279051e7031d service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:19.049427 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2bef5-cdcf-40c1-a94c-279051e7031d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:19.081900 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2bef5-cdcf-40c1-a94c-279051e7031d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:19.089410 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2bef5-cdcf-40c1-a94c-279051e7031d service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:19.119192 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 826/1638] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:20.144996 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b0dce-2b5c-4aa3-b3dc-de93ce9a9a17 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:20.184861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:20.319669 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd09b3-fb1e-44ea-adb8-b20d2586c291 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:20.320749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd09b3-fb1e-44ea-adb8-b20d2586c291 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:20.321438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd09b3-fb1e-44ea-adb8-b20d2586c291 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:20.324240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 827/1639] 10.222.0.60 () {54 vars in 965 bytes} [Mon Sep 19 14:52:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:52:20.331619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b0dce-2b5c-4aa3-b3dc-de93ce9a9a17 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:20.377239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30002a-77af-4695-ae7b-ae8928f1d93b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:20.377239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30002a-77af-4695-ae7b-ae8928f1d93b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:20.377239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30002a-77af-4695-ae7b-ae8928f1d93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:20.377708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b6203-5ad2-47bb-8c68-88c329d11015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:20.377708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b6203-5ad2-47bb-8c68-88c329d11015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:20.377708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b6203-5ad2-47bb-8c68-88c329d11015 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:20.377708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b6203-5ad2-47bb-8c68-88c329d11015 admin admin] PATH_INFO: `/v3/users/9a8f128e3bd8499abea2a71532eb4cf6` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:20.377975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704b6203-5ad2-47bb-8c68-88c329d11015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8f128e3bd8499abea2a71532eb4cf6)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:20.377975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704b6203-5ad2-47bb-8c68-88c329d11015 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:20.404045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2b30002a-77af-4695-ae7b-ae8928f1d93b None None] MFA Rules not processed for user `59b5d2fd362b417c9b9be75f88a667ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:20.405471 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2b30002a-77af-4695-ae7b-ae8928f1d93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:20.428529 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 828/1640] 10.222.0.60 () {58 vars in 1062 bytes} [Mon Sep 19 14:52:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:20.434363 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056624c-13bc-45f0-8f0f-db6943f956aa None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:20.434363 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056624c-13bc-45f0-8f0f-db6943f956aa None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:20.434688 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056624c-13bc-45f0-8f0f-db6943f956aa None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:20.436470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 829/1641] 10.222.0.60 () {54 vars in 912 bytes} [Mon Sep 19 14:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:52:20.442457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5ad67-a1d6-4ba7-89c7-110e07072e0e None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:20.442900 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5ad67-a1d6-4ba7-89c7-110e07072e0e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:20.443282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5ad67-a1d6-4ba7-89c7-110e07072e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:20.492729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-6ad5ad67-a1d6-4ba7-89c7-110e07072e0e None None] MFA Rules not processed for user `59b5d2fd362b417c9b9be75f88a667ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:20.495743 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6ad5ad67-a1d6-4ba7-89c7-110e07072e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:20.520794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 830/1642] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Sep 19 14:52:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:52:20.539529 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5ad67-a1d6-4ba7-89c7-110e07072e0e None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:20.544956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-704b6203-5ad2-47bb-8c68-88c329d11015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8f128e3bd8499abea2a71532eb4cf6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:20.559295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5ad67-a1d6-4ba7-89c7-110e07072e0e None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:20.567641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1518521-a8fe-4b45-8f7c-5c6da9772de9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:20.568452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1518521-a8fe-4b45-8f7c-5c6da9772de9 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:20.568862 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1518521-a8fe-4b45-8f7c-5c6da9772de9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:20.569250 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1518521-a8fe-4b45-8f7c-5c6da9772de9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:20.572847 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-704b6203-5ad2-47bb-8c68-88c329d11015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8f128e3bd8499abea2a71532eb4cf6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:20.600169 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1518521-a8fe-4b45-8f7c-5c6da9772de9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:20.605052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1518521-a8fe-4b45-8f7c-5c6da9772de9 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:20.627241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-704b6203-5ad2-47bb-8c68-88c329d11015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8f128e3bd8499abea2a71532eb4cf6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:20.638747 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 831/1643] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:20.650583 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-704b6203-5ad2-47bb-8c68-88c329d11015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8f128e3bd8499abea2a71532eb4cf6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:20.652777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-704b6203-5ad2-47bb-8c68-88c329d11015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8f128e3bd8499abea2a71532eb4cf6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:20.675305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 813/1644] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:52:20 2022] DELETE /identity/v3/users/9a8f128e3bd8499abea2a71532eb4cf6 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:20.676586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b5590-e8ac-4b98-8a4b-e80df2c6186f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:20.677127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b5590-e8ac-4b98-8a4b-e80df2c6186f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:20.677513 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b5590-e8ac-4b98-8a4b-e80df2c6186f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:20.719558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5d3b5590-e8ac-4b98-8a4b-e80df2c6186f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:20.721341 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5d3b5590-e8ac-4b98-8a4b-e80df2c6186f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:20.864743 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 832/1645] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:20.936233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b6203-5ad2-47bb-8c68-88c329d11015 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:20.938076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-704b6203-5ad2-47bb-8c68-88c329d11015 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:21.079202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704b6203-5ad2-47bb-8c68-88c329d11015 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:21.093556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6836386-b25f-46e9-b5ab-97a40aaa24bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:21.094207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6836386-b25f-46e9-b5ab-97a40aaa24bb service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:21.094491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6836386-b25f-46e9-b5ab-97a40aaa24bb service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:21.094770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6836386-b25f-46e9-b5ab-97a40aaa24bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:21.118770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6836386-b25f-46e9-b5ab-97a40aaa24bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:21.120436 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6836386-b25f-46e9-b5ab-97a40aaa24bb service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:21.148258 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 814/1646] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:21.312614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af87efd-d976-4efb-a98c-9ca42b733769 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:21.312903 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af87efd-d976-4efb-a98c-9ca42b733769 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:21.313182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af87efd-d976-4efb-a98c-9ca42b733769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:21.379216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5af87efd-d976-4efb-a98c-9ca42b733769 None None] MFA Rules not processed for user `ab63f77cf82541feac9f238c301a797a`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:21.382053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5af87efd-d976-4efb-a98c-9ca42b733769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:21.469835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:21.513086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 833/1647] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:21 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:21.517706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6836386-b25f-46e9-b5ab-97a40aaa24bb service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:21.519772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b6836386-b25f-46e9-b5ab-97a40aaa24bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:21.546255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6836386-b25f-46e9-b5ab-97a40aaa24bb service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:21.555492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f121ae-10a3-4be3-bdbe-2bc02d34b0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:21.557657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f121ae-10a3-4be3-bdbe-2bc02d34b0cc admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:21.557934 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f121ae-10a3-4be3-bdbe-2bc02d34b0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:21.559417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f121ae-10a3-4be3-bdbe-2bc02d34b0cc admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:21.561990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f121ae-10a3-4be3-bdbe-2bc02d34b0cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:21.567150 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f121ae-10a3-4be3-bdbe-2bc02d34b0cc admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:21.636026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 815/1648] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:21 2022] POST /identity/v3/projects => generated 403 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:21.640404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af87efd-d976-4efb-a98c-9ca42b733769 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:21.654866 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af87efd-d976-4efb-a98c-9ca42b733769 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:21.751190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b1d15-556d-4b99-93a3-11262b6957b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:21.752473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b1d15-556d-4b99-93a3-11262b6957b5 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:21.753001 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b1d15-556d-4b99-93a3-11262b6957b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:21.753421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b1d15-556d-4b99-93a3-11262b6957b5 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:21.754210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b1d15-556d-4b99-93a3-11262b6957b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:21.757171 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b1d15-556d-4b99-93a3-11262b6957b5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:21.798055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 834/1649] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:21 2022] POST /identity/v3/users => generated 308 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:52:21.798566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f121ae-10a3-4be3-bdbe-2bc02d34b0cc admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:21.816111 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f121ae-10a3-4be3-bdbe-2bc02d34b0cc admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:21.825173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec1696-c131-4b8f-be4f-0453c1352149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:21.825809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec1696-c131-4b8f-be4f-0453c1352149 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:21.826090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec1696-c131-4b8f-be4f-0453c1352149 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:21.826366 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec1696-c131-4b8f-be4f-0453c1352149 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:21.827908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec1696-c131-4b8f-be4f-0453c1352149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:21.830124 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec1696-c131-4b8f-be4f-0453c1352149 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:21.854870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:21.880289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 816/1650] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:21.881190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b1d15-556d-4b99-93a3-11262b6957b5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:21.895625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b1d15-556d-4b99-93a3-11262b6957b5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:21.909461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895b5d3-2707-41bf-bb53-7abefd40e203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:21.911953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895b5d3-2707-41bf-bb53-7abefd40e203 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:21.913150 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895b5d3-2707-41bf-bb53-7abefd40e203 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:21.913757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895b5d3-2707-41bf-bb53-7abefd40e203 admin admin] PATH_INFO: `/v3/projects/33209bce29e74c68aae29cbb072d0d01/users/edd1e7ae073848a49663256c21e35f96/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:21.957709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895b5d3-2707-41bf-bb53-7abefd40e203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33209bce29e74c68aae29cbb072d0d01, user_id=edd1e7ae073848a49663256c21e35f96, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:21.964915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895b5d3-2707-41bf-bb53-7abefd40e203 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:21.994289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 835/1651] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:21 2022] PUT /identity/v3/projects/33209bce29e74c68aae29cbb072d0d01/users/edd1e7ae073848a49663256c21e35f96/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:21.995360 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec1696-c131-4b8f-be4f-0453c1352149 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:22.013386 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec1696-c131-4b8f-be4f-0453c1352149 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:22.279572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895b5d3-2707-41bf-bb53-7abefd40e203 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:22.282684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb239d-64b0-4349-809e-bfe9b0eee88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:22.283899 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0895b5d3-2707-41bf-bb53-7abefd40e203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:22.284311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb239d-64b0-4349-809e-bfe9b0eee88b admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:22.284829 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb239d-64b0-4349-809e-bfe9b0eee88b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:22.285685 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb239d-64b0-4349-809e-bfe9b0eee88b admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:22.286492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb239d-64b0-4349-809e-bfe9b0eee88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:22.288476 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb239d-64b0-4349-809e-bfe9b0eee88b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:22.307134 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 817/1652] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:22.308450 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb239d-64b0-4349-809e-bfe9b0eee88b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:22.311036 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cfeb239d-64b0-4349-809e-bfe9b0eee88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:22.318132 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895b5d3-2707-41bf-bb53-7abefd40e203 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:22.328127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7d77d-0af2-4be0-b3c1-04141b68b924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:22.341686 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7d77d-0af2-4be0-b3c1-04141b68b924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:22.344065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7d77d-0af2-4be0-b3c1-04141b68b924 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:22.344601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7d77d-0af2-4be0-b3c1-04141b68b924 admin admin] PATH_INFO: `/v3/users/97342c4d7ffc40a29286e2457dc06615` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:22.348695 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7d77d-0af2-4be0-b3c1-04141b68b924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97342c4d7ffc40a29286e2457dc06615)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:22.354375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: WARNING py.warnings [None req-ffa7d77d-0af2-4be0-b3c1-04141b68b924 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 14:52:22.354375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: warnings.warn(msg) Sep 19 14:52:22.354375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: Sep 19 14:52:22.356119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb239d-64b0-4349-809e-bfe9b0eee88b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:22.357641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7d77d-0af2-4be0-b3c1-04141b68b924 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:22.363639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:22.364330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:22.364619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:22.364901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] PATH_INFO: `/v3/projects/e8d827ccb2fe4b3e81e4eef92a0035c5` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:22.374640 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8d827ccb2fe4b3e81e4eef92a0035c5)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:22.375981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: WARNING py.warnings [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 19 14:52:22.375981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: warnings.warn(msg) Sep 19 14:52:22.375981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: Sep 19 14:52:22.381712 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:22.483717 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ffa7d77d-0af2-4be0-b3c1-04141b68b924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97342c4d7ffc40a29286e2457dc06615'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:22.486576 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8d827ccb2fe4b3e81e4eef92a0035c5', 'request_id': 'req-bc44d96a-cec0-4c0b-b61f-2071fa911feb'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:22.497804 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ffa7d77d-0af2-4be0-b3c1-04141b68b924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97342c4d7ffc40a29286e2457dc06615'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:22.503167 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8d827ccb2fe4b3e81e4eef92a0035c5', 'request_id': 'req-bc44d96a-cec0-4c0b-b61f-2071fa911feb'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:22.515184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8d827ccb2fe4b3e81e4eef92a0035c5', 'request_id': 'req-bc44d96a-cec0-4c0b-b61f-2071fa911feb'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:22.526143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ffa7d77d-0af2-4be0-b3c1-04141b68b924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97342c4d7ffc40a29286e2457dc06615'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:22.542175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ffa7d77d-0af2-4be0-b3c1-04141b68b924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97342c4d7ffc40a29286e2457dc06615'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:22.543163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ffa7d77d-0af2-4be0-b3c1-04141b68b924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97342c4d7ffc40a29286e2457dc06615'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:22.546704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] The token cache is being invalidate because project e8d827ccb2fe4b3e81e4eef92a0035c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:52:22.549264 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:22.550632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:22.553228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 818/1653] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:52:22 2022] DELETE /identity/v3/projects/e8d827ccb2fe4b3e81e4eef92a0035c5 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:22.554026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:22.555683 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 836/1654] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:52:22 2022] DELETE /identity/v3/users/97342c4d7ffc40a29286e2457dc06615 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:22.559788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:22.561838 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7d77d-0af2-4be0-b3c1-04141b68b924 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:22.567754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ffa7d77d-0af2-4be0-b3c1-04141b68b924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:22.615806 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:22.722982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44d96a-cec0-4c0b-b61f-2071fa911feb admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:22.725080 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7d77d-0af2-4be0-b3c1-04141b68b924 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:22.728780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3466d-fbc5-46f7-87c3-808402c1358f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:22.730353 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3466d-fbc5-46f7-87c3-808402c1358f admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:22.730497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3466d-fbc5-46f7-87c3-808402c1358f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:22.730665 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74cab2-4508-4815-99de-2574f7b42436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:22.730813 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3466d-fbc5-46f7-87c3-808402c1358f admin admin] PATH_INFO: `/v3/projects/33209bce29e74c68aae29cbb072d0d01/users/edd1e7ae073848a49663256c21e35f96/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:22.731548 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74cab2-4508-4815-99de-2574f7b42436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:22.731919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74cab2-4508-4815-99de-2574f7b42436 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:22.732300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74cab2-4508-4815-99de-2574f7b42436 admin admin] PATH_INFO: `/v3/users/ced4b56fdd1b48afa2eaca18ebce7c0c` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:22.733069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3466d-fbc5-46f7-87c3-808402c1358f admin admin] RBAC: Authorizing `identity:create_grant(project_id=33209bce29e74c68aae29cbb072d0d01, user_id=edd1e7ae073848a49663256c21e35f96, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:22.733792 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74cab2-4508-4815-99de-2574f7b42436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ced4b56fdd1b48afa2eaca18ebce7c0c)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:22.735871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74cab2-4508-4815-99de-2574f7b42436 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:22.736472 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3466d-fbc5-46f7-87c3-808402c1358f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:22.755492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 819/1655] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:22 2022] PUT /identity/v3/projects/33209bce29e74c68aae29cbb072d0d01/users/edd1e7ae073848a49663256c21e35f96/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:22.760801 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b6697-d390-4a5a-84bd-f88fef9268f5 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:22.761101 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b6697-d390-4a5a-84bd-f88fef9268f5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:22.761385 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19b6697-d390-4a5a-84bd-f88fef9268f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:22.812170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e19b6697-d390-4a5a-84bd-f88fef9268f5 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:22.816638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e19b6697-d390-4a5a-84bd-f88fef9268f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:22.819856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-de74cab2-4508-4815-99de-2574f7b42436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ced4b56fdd1b48afa2eaca18ebce7c0c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:22.827981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:22.830577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-de74cab2-4508-4815-99de-2574f7b42436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ced4b56fdd1b48afa2eaca18ebce7c0c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:22.848358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-de74cab2-4508-4815-99de-2574f7b42436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ced4b56fdd1b48afa2eaca18ebce7c0c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:22.863286 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-de74cab2-4508-4815-99de-2574f7b42436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ced4b56fdd1b48afa2eaca18ebce7c0c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:22.864674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-de74cab2-4508-4815-99de-2574f7b42436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ced4b56fdd1b48afa2eaca18ebce7c0c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:22.885142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 837/1656] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:52:22 2022] DELETE /identity/v3/users/ced4b56fdd1b48afa2eaca18ebce7c0c => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:22.941820 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 820/1657] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:23.053549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74cab2-4508-4815-99de-2574f7b42436 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:23.055544 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-de74cab2-4508-4815-99de-2574f7b42436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:23.185665 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74cab2-4508-4815-99de-2574f7b42436 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:23.196097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962caf5-55da-4cc5-af1f-6d00a69d9f1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:23.197902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962caf5-55da-4cc5-af1f-6d00a69d9f1e service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:23.198491 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962caf5-55da-4cc5-af1f-6d00a69d9f1e service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:23.199138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962caf5-55da-4cc5-af1f-6d00a69d9f1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:23.317556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:23.351235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962caf5-55da-4cc5-af1f-6d00a69d9f1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:23.356405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962caf5-55da-4cc5-af1f-6d00a69d9f1e service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:23.387781 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 838/1658] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:25.251153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b6697-d390-4a5a-84bd-f88fef9268f5 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:25.252919 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e19b6697-d390-4a5a-84bd-f88fef9268f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:25.282297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19b6697-d390-4a5a-84bd-f88fef9268f5 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:25.308330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471fb9f-603e-4a21-aae6-8c2cc344c54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:25.309180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471fb9f-603e-4a21-aae6-8c2cc344c54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:25.309479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471fb9f-603e-4a21-aae6-8c2cc344c54f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:25.309825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471fb9f-603e-4a21-aae6-8c2cc344c54f admin admin] PATH_INFO: `/v3/projects/bd410f7316a740d1b309a39dcf9dfe2e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:25.313164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471fb9f-603e-4a21-aae6-8c2cc344c54f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd410f7316a740d1b309a39dcf9dfe2e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:25.315242 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471fb9f-603e-4a21-aae6-8c2cc344c54f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:25.519756 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8471fb9f-603e-4a21-aae6-8c2cc344c54f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd410f7316a740d1b309a39dcf9dfe2e', 'request_id': 'req-8471fb9f-603e-4a21-aae6-8c2cc344c54f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:25.539519 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8471fb9f-603e-4a21-aae6-8c2cc344c54f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd410f7316a740d1b309a39dcf9dfe2e', 'request_id': 'req-8471fb9f-603e-4a21-aae6-8c2cc344c54f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:25.539709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e1d1b-b9be-46a9-8aa0-1c9b667a577d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:25.540886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e1d1b-b9be-46a9-8aa0-1c9b667a577d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:25.540886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e1d1b-b9be-46a9-8aa0-1c9b667a577d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:25.574027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8471fb9f-603e-4a21-aae6-8c2cc344c54f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd410f7316a740d1b309a39dcf9dfe2e', 'request_id': 'req-8471fb9f-603e-4a21-aae6-8c2cc344c54f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:25.604395 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8471fb9f-603e-4a21-aae6-8c2cc344c54f admin admin] The token cache is being invalidate because project bd410f7316a740d1b309a39dcf9dfe2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:52:25.607219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8471fb9f-603e-4a21-aae6-8c2cc344c54f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:25.609681 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8471fb9f-603e-4a21-aae6-8c2cc344c54f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:25.614505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 821/1659] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:52:25 2022] DELETE /identity/v3/projects/bd410f7316a740d1b309a39dcf9dfe2e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:25.619427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471fb9f-603e-4a21-aae6-8c2cc344c54f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:25.623344 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b38e1d1b-b9be-46a9-8aa0-1c9b667a577d None None] MFA Rules not processed for user `edd1e7ae073848a49663256c21e35f96`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:25.624423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8471fb9f-603e-4a21-aae6-8c2cc344c54f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:25.628486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b38e1d1b-b9be-46a9-8aa0-1c9b667a577d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:25.804095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471fb9f-603e-4a21-aae6-8c2cc344c54f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:25.812379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14442a-0a4b-444c-b167-0cdab23f1473 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:25.813347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14442a-0a4b-444c-b167-0cdab23f1473 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:25.813633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14442a-0a4b-444c-b167-0cdab23f1473 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:25.813911 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14442a-0a4b-444c-b167-0cdab23f1473 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:25.821506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ce14442a-0a4b-444c-b167-0cdab23f1473 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:25.849950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 839/1660] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:25 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:25.850976 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e1d1b-b9be-46a9-8aa0-1c9b667a577d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:25.854735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b38e1d1b-b9be-46a9-8aa0-1c9b667a577d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:25.957532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:26.021206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14442a-0a4b-444c-b167-0cdab23f1473 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:26.022762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14442a-0a4b-444c-b167-0cdab23f1473 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:26.038013 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e1d1b-b9be-46a9-8aa0-1c9b667a577d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:26.048370 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70d364-0a5c-46c4-9eb5-08465dd68fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:26.049267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70d364-0a5c-46c4-9eb5-08465dd68fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:26.049719 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70d364-0a5c-46c4-9eb5-08465dd68fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:26.050056 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70d364-0a5c-46c4-9eb5-08465dd68fd9 admin admin] PATH_INFO: `/v3/projects/505a2b7bf4e64b689bd8081324cb729e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:26.054971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70d364-0a5c-46c4-9eb5-08465dd68fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=505a2b7bf4e64b689bd8081324cb729e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:26.061718 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70d364-0a5c-46c4-9eb5-08465dd68fd9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:26.071068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 822/1661] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:26.074224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2e0e1-ae71-469b-b40f-f7441507ee7a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:26.074525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2e0e1-ae71-469b-b40f-f7441507ee7a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:26.074810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2e0e1-ae71-469b-b40f-f7441507ee7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:26.168424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-91e2e0e1-ae71-469b-b40f-f7441507ee7a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:26.169186 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-91e2e0e1-ae71-469b-b40f-f7441507ee7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:26.198098 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 823/1662] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:26.212498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aad2cd-9d2b-4fd8-9054-e726af61cb42 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:26.213964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aad2cd-9d2b-4fd8-9054-e726af61cb42 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:26.216051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aad2cd-9d2b-4fd8-9054-e726af61cb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:26.248881 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9f70d364-0a5c-46c4-9eb5-08465dd68fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '505a2b7bf4e64b689bd8081324cb729e', 'request_id': 'req-9f70d364-0a5c-46c4-9eb5-08465dd68fd9'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:26.261393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9f70d364-0a5c-46c4-9eb5-08465dd68fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '505a2b7bf4e64b689bd8081324cb729e', 'request_id': 'req-9f70d364-0a5c-46c4-9eb5-08465dd68fd9'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:26.269235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-89aad2cd-9d2b-4fd8-9054-e726af61cb42 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:26.270966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-89aad2cd-9d2b-4fd8-9054-e726af61cb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:26.286242 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9f70d364-0a5c-46c4-9eb5-08465dd68fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '505a2b7bf4e64b689bd8081324cb729e', 'request_id': 'req-9f70d364-0a5c-46c4-9eb5-08465dd68fd9'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:26.314275 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:26.327988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9f70d364-0a5c-46c4-9eb5-08465dd68fd9 admin admin] The token cache is being invalidate because project 505a2b7bf4e64b689bd8081324cb729e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:52:26.329358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9f70d364-0a5c-46c4-9eb5-08465dd68fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:26.333954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9f70d364-0a5c-46c4-9eb5-08465dd68fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:26.341505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 840/1663] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:52:25 2022] DELETE /identity/v3/projects/505a2b7bf4e64b689bd8081324cb729e => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:26.343481 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a9f06-8300-4571-b0d0-fde40b067878 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:26.344080 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a9f06-8300-4571-b0d0-fde40b067878 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:26.344080 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a9f06-8300-4571-b0d0-fde40b067878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:26.389615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c36a9f06-8300-4571-b0d0-fde40b067878 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:26.390856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c36a9f06-8300-4571-b0d0-fde40b067878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:26.402303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 824/1664] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:26.406598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aad2cd-9d2b-4fd8-9054-e726af61cb42 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:26.414291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 841/1665] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:26.416933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131fcb4-cddb-45cd-b777-b6385cd6ee55 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:26.417225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131fcb4-cddb-45cd-b777-b6385cd6ee55 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:26.417507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131fcb4-cddb-45cd-b777-b6385cd6ee55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:26.424133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aad2cd-9d2b-4fd8-9054-e726af61cb42 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:26.434535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c51257-05ac-42cb-8b04-63ec29552ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:26.435271 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c51257-05ac-42cb-8b04-63ec29552ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:26.436251 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c51257-05ac-42cb-8b04-63ec29552ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:26.436634 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c51257-05ac-42cb-8b04-63ec29552ce0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:26.437512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c51257-05ac-42cb-8b04-63ec29552ce0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:52:26.438115 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c51257-05ac-42cb-8b04-63ec29552ce0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:26.443066 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c51257-05ac-42cb-8b04-63ec29552ce0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:26.468865 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2131fcb4-cddb-45cd-b777-b6385cd6ee55 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:26.470352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2131fcb4-cddb-45cd-b777-b6385cd6ee55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:26.470782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 825/1666] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:52:26.480480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c51257-05ac-42cb-8b04-63ec29552ce0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:26.495416 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 842/1667] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:26.499737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131fcb4-cddb-45cd-b777-b6385cd6ee55 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:26.511897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c51257-05ac-42cb-8b04-63ec29552ce0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:26.514660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131fcb4-cddb-45cd-b777-b6385cd6ee55 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:26.523749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2daa2-ff8e-40c2-b7a4-ec3fe0498ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:26.524625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2daa2-ff8e-40c2-b7a4-ec3fe0498ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:26.525064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2daa2-ff8e-40c2-b7a4-ec3fe0498ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:26.525590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2daa2-ff8e-40c2-b7a4-ec3fe0498ea2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:26.526483 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2daa2-ff8e-40c2-b7a4-ec3fe0498ea2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:52:26.526932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2daa2-ff8e-40c2-b7a4-ec3fe0498ea2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:26.528708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306c58f-872b-443e-95f9-44b4e1b2720f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:26.529497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306c58f-872b-443e-95f9-44b4e1b2720f admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:26.529775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2daa2-ff8e-40c2-b7a4-ec3fe0498ea2 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:26.530165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306c58f-872b-443e-95f9-44b4e1b2720f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:26.531310 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306c58f-872b-443e-95f9-44b4e1b2720f admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:26.532086 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306c58f-872b-443e-95f9-44b4e1b2720f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:26.553308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306c58f-872b-443e-95f9-44b4e1b2720f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:26.575170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 843/1668] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:52:26.579985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2daa2-ff8e-40c2-b7a4-ec3fe0498ea2 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:26.605555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2daa2-ff8e-40c2-b7a4-ec3fe0498ea2 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:26.631686 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab23073-124d-4e47-9349-d663f60b0832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:26.632565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 826/1669] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:26 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:26.636934 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306c58f-872b-443e-95f9-44b4e1b2720f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:26.639029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab23073-124d-4e47-9349-d663f60b0832 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:26.640656 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab23073-124d-4e47-9349-d663f60b0832 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:26.640949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab23073-124d-4e47-9349-d663f60b0832 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:26.641602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab23073-124d-4e47-9349-d663f60b0832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:26.647116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab23073-124d-4e47-9349-d663f60b0832 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:26.713526 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 844/1670] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:26 2022] POST /identity/v3/projects => generated 403 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:26.717673 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab23073-124d-4e47-9349-d663f60b0832 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:26.778563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306c58f-872b-443e-95f9-44b4e1b2720f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:26.862135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab23073-124d-4e47-9349-d663f60b0832 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:26.876514 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aefe41-cc0d-4c48-91dc-ad1675f662de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:26.877692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aefe41-cc0d-4c48-91dc-ad1675f662de admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:26.878286 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aefe41-cc0d-4c48-91dc-ad1675f662de admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:26.878852 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aefe41-cc0d-4c48-91dc-ad1675f662de admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:26.879667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aefe41-cc0d-4c48-91dc-ad1675f662de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:26.883267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aefe41-cc0d-4c48-91dc-ad1675f662de admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:26.895437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:26.930542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38ffe2-9372-4460-a123-c88cf698f99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:26.931578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38ffe2-9372-4460-a123-c88cf698f99b admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:26.932013 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38ffe2-9372-4460-a123-c88cf698f99b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:26.932382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e38ffe2-9372-4460-a123-c88cf698f99b admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:26.933197 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38ffe2-9372-4460-a123-c88cf698f99b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:26.935370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e38ffe2-9372-4460-a123-c88cf698f99b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:26.937042 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 845/1671] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:26 2022] POST /identity/v3/users => generated 302 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:52:26.941609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aefe41-cc0d-4c48-91dc-ad1675f662de admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:26.960202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aefe41-cc0d-4c48-91dc-ad1675f662de admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:26.975314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31127d76-0f35-4859-a76c-ee171e4271b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:26.975865 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31127d76-0f35-4859-a76c-ee171e4271b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:26.976145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31127d76-0f35-4859-a76c-ee171e4271b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:26.976837 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31127d76-0f35-4859-a76c-ee171e4271b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:26.977106 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31127d76-0f35-4859-a76c-ee171e4271b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:26.982758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31127d76-0f35-4859-a76c-ee171e4271b7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:26.992405 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 827/1672] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:26 2022] POST /identity/v3/users => generated 290 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:26.997703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38ffe2-9372-4460-a123-c88cf698f99b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:27.022308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e38ffe2-9372-4460-a123-c88cf698f99b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:27.027960 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 846/1673] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:52:27.033823 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31127d76-0f35-4859-a76c-ee171e4271b7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:27.034815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532c770-80b7-4b27-a002-d2717b45196f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:27.035563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532c770-80b7-4b27-a002-d2717b45196f admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:27.035974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532c770-80b7-4b27-a002-d2717b45196f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:27.036346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4532c770-80b7-4b27-a002-d2717b45196f admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:27.037218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532c770-80b7-4b27-a002-d2717b45196f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:27.041068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4532c770-80b7-4b27-a002-d2717b45196f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:27.069012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 828/1674] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:52:27.070209 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31127d76-0f35-4859-a76c-ee171e4271b7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:27.075573 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532c770-80b7-4b27-a002-d2717b45196f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:27.084620 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf005fe-0aac-4d47-a259-053e7b5603a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:27.086455 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf005fe-0aac-4d47-a259-053e7b5603a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:27.086874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf005fe-0aac-4d47-a259-053e7b5603a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:27.094191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf005fe-0aac-4d47-a259-053e7b5603a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:27.095091 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf005fe-0aac-4d47-a259-053e7b5603a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:27.097778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf005fe-0aac-4d47-a259-053e7b5603a3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:27.106368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4532c770-80b7-4b27-a002-d2717b45196f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:27.115617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4396000-58fa-4bbf-b3ac-53eb665a947a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:27.116416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4396000-58fa-4bbf-b3ac-53eb665a947a admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:27.116792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4396000-58fa-4bbf-b3ac-53eb665a947a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:27.117161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4396000-58fa-4bbf-b3ac-53eb665a947a admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:27.117902 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4396000-58fa-4bbf-b3ac-53eb665a947a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:27.121867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4396000-58fa-4bbf-b3ac-53eb665a947a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:27.134831 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 847/1675] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:27.140942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf005fe-0aac-4d47-a259-053e7b5603a3 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:27.150528 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 829/1676] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:27.155523 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4396000-58fa-4bbf-b3ac-53eb665a947a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:27.160127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf005fe-0aac-4d47-a259-053e7b5603a3 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:27.178074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4396000-58fa-4bbf-b3ac-53eb665a947a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:27.192367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1698e-2833-4acc-87fc-32779e73e7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:27.193242 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1698e-2833-4acc-87fc-32779e73e7d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:27.193354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556c942-6cd8-4e81-be4e-76ac26221446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:27.193711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1698e-2833-4acc-87fc-32779e73e7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:27.194055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556c942-6cd8-4e81-be4e-76ac26221446 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:27.194167 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1698e-2833-4acc-87fc-32779e73e7d5 admin admin] PATH_INFO: `/v3/projects/09228199bc084a8c89bd14aa0658079a/users/26cbf33d505641c29f95f20f29a126da/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:27.194337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556c942-6cd8-4e81-be4e-76ac26221446 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:27.194613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556c942-6cd8-4e81-be4e-76ac26221446 admin admin] PATH_INFO: `/v3/projects/c75feb7246f543b2aab610022f5f1d8d/users/3336c8c5cf2e408191449c2a6499c9b4/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:27.226320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1698e-2833-4acc-87fc-32779e73e7d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09228199bc084a8c89bd14aa0658079a, user_id=26cbf33d505641c29f95f20f29a126da, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:27.229826 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1698e-2833-4acc-87fc-32779e73e7d5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:27.233334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556c942-6cd8-4e81-be4e-76ac26221446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c75feb7246f543b2aab610022f5f1d8d, user_id=3336c8c5cf2e408191449c2a6499c9b4, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:27.236060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556c942-6cd8-4e81-be4e-76ac26221446 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:27.265682 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 830/1677] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:27 2022] PUT /identity/v3/projects/09228199bc084a8c89bd14aa0658079a/users/26cbf33d505641c29f95f20f29a126da/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:27.278101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 848/1678] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:27 2022] PUT /identity/v3/projects/c75feb7246f543b2aab610022f5f1d8d/users/3336c8c5cf2e408191449c2a6499c9b4/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:27.278281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff8a7d-84ad-4e77-8a7f-d3b94bf5959c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:27.278281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff8a7d-84ad-4e77-8a7f-d3b94bf5959c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:27.278281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff8a7d-84ad-4e77-8a7f-d3b94bf5959c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:27.280478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c09a1-69dd-432d-94fb-52b697201898 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:27.280478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c09a1-69dd-432d-94fb-52b697201898 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:27.280478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c09a1-69dd-432d-94fb-52b697201898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:27.357789 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1c1c09a1-69dd-432d-94fb-52b697201898 None None] MFA Rules not processed for user `3336c8c5cf2e408191449c2a6499c9b4`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:27.362310 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1c1c09a1-69dd-432d-94fb-52b697201898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:27.389915 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1eff8a7d-84ad-4e77-8a7f-d3b94bf5959c None None] MFA Rules not processed for user `26cbf33d505641c29f95f20f29a126da`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:27.391347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1eff8a7d-84ad-4e77-8a7f-d3b94bf5959c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:27.480471 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:27.548928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 849/1679] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:27 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:52:27.557261 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c09a1-69dd-432d-94fb-52b697201898 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:27.581279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 831/1680] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:27.588348 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff8a7d-84ad-4e77-8a7f-d3b94bf5959c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:27.741014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff8a7d-84ad-4e77-8a7f-d3b94bf5959c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:27.750020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c09a1-69dd-432d-94fb-52b697201898 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:27.757768 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4eb12-ab94-43a2-96c3-206380f0a40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:27.758605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4eb12-ab94-43a2-96c3-206380f0a40a admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:27.759519 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4eb12-ab94-43a2-96c3-206380f0a40a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:27.759966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4eb12-ab94-43a2-96c3-206380f0a40a admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:27.760695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d4eb12-ab94-43a2-96c3-206380f0a40a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:27.767217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a1300-6e31-4e70-b114-c8ad5ce69391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:27.768672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a1300-6e31-4e70-b114-c8ad5ce69391 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:27.769284 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a1300-6e31-4e70-b114-c8ad5ce69391 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:27.769733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d4eb12-ab94-43a2-96c3-206380f0a40a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:27.770107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a1300-6e31-4e70-b114-c8ad5ce69391 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:27.771290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a1300-6e31-4e70-b114-c8ad5ce69391 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:27.785736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a1300-6e31-4e70-b114-c8ad5ce69391 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:27.849025 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 850/1681] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:27 2022] POST /identity/v3/projects => generated 403 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:27.853565 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a1300-6e31-4e70-b114-c8ad5ce69391 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:27.861305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 832/1682] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:27 2022] POST /identity/v3/projects => generated 381 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:52:27.865495 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4eb12-ab94-43a2-96c3-206380f0a40a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:27.889996 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:28.025249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a1300-6e31-4e70-b114-c8ad5ce69391 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:28.063598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4eb12-ab94-43a2-96c3-206380f0a40a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:28.081971 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb69f2f-f2f2-49a0-a3a2-5e590ddcff0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:28.087677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb69f2f-f2f2-49a0-a3a2-5e590ddcff0a admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:28.088286 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb69f2f-f2f2-49a0-a3a2-5e590ddcff0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:28.088728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb69f2f-f2f2-49a0-a3a2-5e590ddcff0a admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:28.089782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb69f2f-f2f2-49a0-a3a2-5e590ddcff0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:28.091997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb69f2f-f2f2-49a0-a3a2-5e590ddcff0a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:28.137126 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 833/1683] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:27 2022] POST /identity/v3/users => generated 297 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:28.141615 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb69f2f-f2f2-49a0-a3a2-5e590ddcff0a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:28.161038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb69f2f-f2f2-49a0-a3a2-5e590ddcff0a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:28.175149 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69f0a8-de0c-4334-85f2-9e3eafd59bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:28.175867 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69f0a8-de0c-4334-85f2-9e3eafd59bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:28.176160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69f0a8-de0c-4334-85f2-9e3eafd59bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:28.176437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69f0a8-de0c-4334-85f2-9e3eafd59bb9 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:28.177640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69f0a8-de0c-4334-85f2-9e3eafd59bb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:28.178276 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb13b0-4289-49a5-88f1-0d87f719496c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:28.179096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb13b0-4289-49a5-88f1-0d87f719496c admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:28.179548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb13b0-4289-49a5-88f1-0d87f719496c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:28.180027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb13b0-4289-49a5-88f1-0d87f719496c admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:28.180883 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb13b0-4289-49a5-88f1-0d87f719496c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:28.181735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69f0a8-de0c-4334-85f2-9e3eafd59bb9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:28.183324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb13b0-4289-49a5-88f1-0d87f719496c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:28.208317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 834/1684] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:52:28.214628 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb13b0-4289-49a5-88f1-0d87f719496c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:28.235812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb13b0-4289-49a5-88f1-0d87f719496c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:28.242552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531d86f-4148-4846-98c8-3233a0eeb01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:28.244239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531d86f-4148-4846-98c8-3233a0eeb01b admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:28.244744 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531d86f-4148-4846-98c8-3233a0eeb01b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:28.245332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531d86f-4148-4846-98c8-3233a0eeb01b admin admin] PATH_INFO: `/v3/projects/26bfcd50a403404c96dec2b695857d26/users/bbcd9d15f7d2461083b7e005934ad7c1/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:28.246373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 851/1685] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:27 2022] POST /identity/v3/users => generated 308 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:52:28.247615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69f0a8-de0c-4334-85f2-9e3eafd59bb9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:28.264645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69f0a8-de0c-4334-85f2-9e3eafd59bb9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:28.281546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c0d2a-b216-41cd-a8fa-228f26b1610e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:28.282280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c0d2a-b216-41cd-a8fa-228f26b1610e admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:28.282528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c0d2a-b216-41cd-a8fa-228f26b1610e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:28.282821 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c0d2a-b216-41cd-a8fa-228f26b1610e admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:28.282931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531d86f-4148-4846-98c8-3233a0eeb01b admin admin] RBAC: Authorizing `identity:create_grant(project_id=26bfcd50a403404c96dec2b695857d26, user_id=bbcd9d15f7d2461083b7e005934ad7c1, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:28.283359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c0d2a-b216-41cd-a8fa-228f26b1610e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:28.284972 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c0d2a-b216-41cd-a8fa-228f26b1610e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:28.285705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531d86f-4148-4846-98c8-3233a0eeb01b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:28.286663 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:28.309264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 852/1686] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:28.311433 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c0d2a-b216-41cd-a8fa-228f26b1610e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:28.321448 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 835/1687] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:28 2022] PUT /identity/v3/projects/26bfcd50a403404c96dec2b695857d26/users/bbcd9d15f7d2461083b7e005934ad7c1/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:28.325367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531d86f-4148-4846-98c8-3233a0eeb01b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:28.331303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c0d2a-b216-41cd-a8fa-228f26b1610e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:28.456963 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46914b-1efc-487a-823b-dbe8f1bd0f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:28.457652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46914b-1efc-487a-823b-dbe8f1bd0f79 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:28.457933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46914b-1efc-487a-823b-dbe8f1bd0f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:28.458208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46914b-1efc-487a-823b-dbe8f1bd0f79 admin admin] PATH_INFO: `/v3/projects/e5676c2d31e7404eb0045d67130563ae/users/cd20577980ca46c5bb74879061aee21f/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:28.463123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531d86f-4148-4846-98c8-3233a0eeb01b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:28.469097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17953fa3-434a-403d-8b18-2c99efc33840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:28.469097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953fa3-434a-403d-8b18-2c99efc33840 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:28.469097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953fa3-434a-403d-8b18-2c99efc33840 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:28.469441 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17953fa3-434a-403d-8b18-2c99efc33840 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:28.469859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17953fa3-434a-403d-8b18-2c99efc33840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:28.471460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17953fa3-434a-403d-8b18-2c99efc33840 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:28.490182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46914b-1efc-487a-823b-dbe8f1bd0f79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5676c2d31e7404eb0045d67130563ae, user_id=cd20577980ca46c5bb74879061aee21f, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:28.493023 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46914b-1efc-487a-823b-dbe8f1bd0f79 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:28.502467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 836/1688] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:28.503657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17953fa3-434a-403d-8b18-2c99efc33840 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:28.519363 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 853/1689] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:28 2022] PUT /identity/v3/projects/e5676c2d31e7404eb0045d67130563ae/users/cd20577980ca46c5bb74879061aee21f/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:52:28.522555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17953fa3-434a-403d-8b18-2c99efc33840 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:28.524240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46914b-1efc-487a-823b-dbe8f1bd0f79 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:28.658198 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30699fd5-0341-485d-9aea-7899aef5f046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:28.658887 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30699fd5-0341-485d-9aea-7899aef5f046 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:28.659168 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30699fd5-0341-485d-9aea-7899aef5f046 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:28.659454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30699fd5-0341-485d-9aea-7899aef5f046 admin admin] PATH_INFO: `/v3/projects/26bfcd50a403404c96dec2b695857d26/users/bbcd9d15f7d2461083b7e005934ad7c1/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:28.662910 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30699fd5-0341-485d-9aea-7899aef5f046 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26bfcd50a403404c96dec2b695857d26, user_id=bbcd9d15f7d2461083b7e005934ad7c1, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:28.665742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30699fd5-0341-485d-9aea-7899aef5f046 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:28.673253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:28.676392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46914b-1efc-487a-823b-dbe8f1bd0f79 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:28.684065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83e7a4-1a71-498e-829c-98fa0218ad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:28.684722 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83e7a4-1a71-498e-829c-98fa0218ad98 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:28.685148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83e7a4-1a71-498e-829c-98fa0218ad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:28.685437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83e7a4-1a71-498e-829c-98fa0218ad98 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:28.686115 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83e7a4-1a71-498e-829c-98fa0218ad98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:28.687893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83e7a4-1a71-498e-829c-98fa0218ad98 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:28.704450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 854/1690] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:52:28.705551 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83e7a4-1a71-498e-829c-98fa0218ad98 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:28.705780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 837/1691] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:28 2022] PUT /identity/v3/projects/26bfcd50a403404c96dec2b695857d26/users/bbcd9d15f7d2461083b7e005934ad7c1/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:52:28.717516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff11342-dc5f-4a4e-a44b-2905bc548be4 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:28.717516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff11342-dc5f-4a4e-a44b-2905bc548be4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:28.717516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff11342-dc5f-4a4e-a44b-2905bc548be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:28.724695 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83e7a4-1a71-498e-829c-98fa0218ad98 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:28.788273 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-eff11342-dc5f-4a4e-a44b-2905bc548be4 None None] MFA Rules not processed for user `bbcd9d15f7d2461083b7e005934ad7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:28.789499 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-eff11342-dc5f-4a4e-a44b-2905bc548be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:28.883265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475db91e-0ed3-4351-ac10-553274f43abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:28.884550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475db91e-0ed3-4351-ac10-553274f43abe admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:28.885440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475db91e-0ed3-4351-ac10-553274f43abe admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:28.885440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475db91e-0ed3-4351-ac10-553274f43abe admin admin] PATH_INFO: `/v3/projects/e5676c2d31e7404eb0045d67130563ae/users/cd20577980ca46c5bb74879061aee21f/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:28.893252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475db91e-0ed3-4351-ac10-553274f43abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5676c2d31e7404eb0045d67130563ae, user_id=cd20577980ca46c5bb74879061aee21f, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:28.897370 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475db91e-0ed3-4351-ac10-553274f43abe admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:28.932522 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 855/1692] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:28 2022] PUT /identity/v3/projects/e5676c2d31e7404eb0045d67130563ae/users/cd20577980ca46c5bb74879061aee21f/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:28.934062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bf236-9b19-4ac0-9654-50c077796da2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:28.934358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bf236-9b19-4ac0-9654-50c077796da2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:28.934639 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bf236-9b19-4ac0-9654-50c077796da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:28.951162 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 838/1693] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:52:28.967091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff11342-dc5f-4a4e-a44b-2905bc548be4 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:28.969319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-eff11342-dc5f-4a4e-a44b-2905bc548be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:28.979907 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:29.003093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-289bf236-9b19-4ac0-9654-50c077796da2 None None] MFA Rules not processed for user `cd20577980ca46c5bb74879061aee21f`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:29.020613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-289bf236-9b19-4ac0-9654-50c077796da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:29.088258 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff11342-dc5f-4a4e-a44b-2905bc548be4 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:29.194822 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563d64e-dfe3-4a50-8ae5-6ab635824b6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:29.195504 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563d64e-dfe3-4a50-8ae5-6ab635824b6b service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:29.196149 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563d64e-dfe3-4a50-8ae5-6ab635824b6b service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:29.196149 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563d64e-dfe3-4a50-8ae5-6ab635824b6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:29.229289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 856/1694] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:28 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:29.234802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bf236-9b19-4ac0-9654-50c077796da2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:29.238334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-289bf236-9b19-4ac0-9654-50c077796da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:29.261743 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bf236-9b19-4ac0-9654-50c077796da2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:29.270884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bcafe8-64f4-4204-abbf-b8289e55fa30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:29.271683 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcafe8-64f4-4204-abbf-b8289e55fa30 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:29.272098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcafe8-64f4-4204-abbf-b8289e55fa30 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:29.272547 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bcafe8-64f4-4204-abbf-b8289e55fa30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:29.288205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcafe8-64f4-4204-abbf-b8289e55fa30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:29.290109 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bcafe8-64f4-4204-abbf-b8289e55fa30 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:29.298345 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:29.319236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563d64e-dfe3-4a50-8ae5-6ab635824b6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:29.324941 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 857/1695] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:29 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:29.338282 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563d64e-dfe3-4a50-8ae5-6ab635824b6b service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:29.353780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 839/1696] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:52:29.583135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a5545-6bb7-4d1b-a85e-9d67cb78bf2d None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:29.583754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a5545-6bb7-4d1b-a85e-9d67cb78bf2d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:29.585247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a5545-6bb7-4d1b-a85e-9d67cb78bf2d None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:29.587912 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 858/1697] 10.222.0.60 () {54 vars in 921 bytes} [Mon Sep 19 14:52:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:52:29.607007 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c51a2d-bba8-4efd-8a36-21d475abb1a6 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:29.607304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c51a2d-bba8-4efd-8a36-21d475abb1a6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:29.607621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c51a2d-bba8-4efd-8a36-21d475abb1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:29.645154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-24c51a2d-bba8-4efd-8a36-21d475abb1a6 None None] MFA Rules not processed for user `568189358b154b66a729fae2f0411d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:29.646403 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-24c51a2d-bba8-4efd-8a36-21d475abb1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:29.833429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aeb738-c09b-40d3-be3f-62c6eda031e2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:29.833710 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aeb738-c09b-40d3-be3f-62c6eda031e2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:29.833985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aeb738-c09b-40d3-be3f-62c6eda031e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:29.869607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 840/1698] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Sep 19 14:52:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:52:29.911333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-89aeb738-c09b-40d3-be3f-62c6eda031e2 None None] MFA Rules not processed for user `b2c40600d2fd4eea8e4ba999050abd73`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:29.912598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-89aeb738-c09b-40d3-be3f-62c6eda031e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:29.995392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c51a2d-bba8-4efd-8a36-21d475abb1a6 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:29.996904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-24c51a2d-bba8-4efd-8a36-21d475abb1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:30.029704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c51a2d-bba8-4efd-8a36-21d475abb1a6 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:30.048598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dba7b2-0664-461a-aa7f-68fc9815ad2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '568189358b154b66a729fae2f0411d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:30.049248 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dba7b2-0664-461a-aa7f-68fc9815ad2d service placement] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:30.049525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dba7b2-0664-461a-aa7f-68fc9815ad2d service placement] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:30.049791 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dba7b2-0664-461a-aa7f-68fc9815ad2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:30.091114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dba7b2-0664-461a-aa7f-68fc9815ad2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:30.092920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dba7b2-0664-461a-aa7f-68fc9815ad2d service placement] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:30.133246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 841/1699] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:30.148444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 859/1700] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:30.168905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dba7b2-0664-461a-aa7f-68fc9815ad2d service placement] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:30.170402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e3dba7b2-0664-461a-aa7f-68fc9815ad2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:30.219439 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34fd1d-e42e-467a-b631-1e8e29432f79 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:30.219998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34fd1d-e42e-467a-b631-1e8e29432f79 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:30.220267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34fd1d-e42e-467a-b631-1e8e29432f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:30.294576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8c34fd1d-e42e-467a-b631-1e8e29432f79 None None] MFA Rules not processed for user `1cccf354894c4311b7710d8a65682eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:30.300682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8c34fd1d-e42e-467a-b631-1e8e29432f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:30.390796 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dba7b2-0664-461a-aa7f-68fc9815ad2d service placement] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:30.401296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f8c95-6f93-4d46-8c87-242098d59d4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:30.402439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f8c95-6f93-4d46-8c87-242098d59d4b service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:30.403031 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f8c95-6f93-4d46-8c87-242098d59d4b service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:30.405160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f8c95-6f93-4d46-8c87-242098d59d4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:30.424141 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26f8c95-6f93-4d46-8c87-242098d59d4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:30.430843 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26f8c95-6f93-4d46-8c87-242098d59d4b service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:30.450320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 842/1701] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:30 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:30.452749 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f8c95-6f93-4d46-8c87-242098d59d4b service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:30.463880 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:30.489786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 860/1702] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:30.491352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34fd1d-e42e-467a-b631-1e8e29432f79 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:30.529728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34fd1d-e42e-467a-b631-1e8e29432f79 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:30.542975 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9f660-7913-4883-9032-948e02cd120c tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbcd9d15f7d2461083b7e005934ad7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26bfcd50a403404c96dec2b695857d26', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:30.543623 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9f660-7913-4883-9032-948e02cd120c tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:30.543937 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9f660-7913-4883-9032-948e02cd120c tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:30.544224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9f660-7913-4883-9032-948e02cd120c tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:30.552602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9f660-7913-4883-9032-948e02cd120c tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:52:30.552602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9f660-7913-4883-9032-948e02cd120c tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:30.555093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc9f660-7913-4883-9032-948e02cd120c tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:30.590324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:30.598003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 861/1703] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:52:30.598003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a4626-c2a7-4dc9-8e60-8de56c1fdbc0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:30.598003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a4626-c2a7-4dc9-8e60-8de56c1fdbc0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:30.599230 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a4626-c2a7-4dc9-8e60-8de56c1fdbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:30.630103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f8c95-6f93-4d46-8c87-242098d59d4b service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:30.645806 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3078e-f6b6-491c-a62b-bd8ca6d5a19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:30.646463 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3078e-f6b6-491c-a62b-bd8ca6d5a19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:30.646905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3078e-f6b6-491c-a62b-bd8ca6d5a19f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:30.647187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c3078e-f6b6-491c-a62b-bd8ca6d5a19f admin admin] PATH_INFO: `/v3/users/3336c8c5cf2e408191449c2a6499c9b4` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:30.648928 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3078e-f6b6-491c-a62b-bd8ca6d5a19f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3336c8c5cf2e408191449c2a6499c9b4)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:30.651687 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c3078e-f6b6-491c-a62b-bd8ca6d5a19f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:30.659608 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f43a4626-c2a7-4dc9-8e60-8de56c1fdbc0 None None] MFA Rules not processed for user `1cccf354894c4311b7710d8a65682eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:30.664520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f43a4626-c2a7-4dc9-8e60-8de56c1fdbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:30.684181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 862/1704] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:30.686557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a4626-c2a7-4dc9-8e60-8de56c1fdbc0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:30.713748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a4626-c2a7-4dc9-8e60-8de56c1fdbc0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:30.729801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544611f0-0177-40bc-9597-0198c614fcca tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbcd9d15f7d2461083b7e005934ad7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26bfcd50a403404c96dec2b695857d26', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:30.730481 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544611f0-0177-40bc-9597-0198c614fcca tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:30.730924 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544611f0-0177-40bc-9597-0198c614fcca tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:30.731208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544611f0-0177-40bc-9597-0198c614fcca tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:30.732066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544611f0-0177-40bc-9597-0198c614fcca tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:30.734704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544611f0-0177-40bc-9597-0198c614fcca tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:30.801068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 863/1705] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:30 2022] POST /identity/v3/projects => generated 355 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:30.802113 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544611f0-0177-40bc-9597-0198c614fcca tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:30.803198 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-98c3078e-f6b6-491c-a62b-bd8ca6d5a19f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3336c8c5cf2e408191449c2a6499c9b4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:30.812610 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-544611f0-0177-40bc-9597-0198c614fcca tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:30.825665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-98c3078e-f6b6-491c-a62b-bd8ca6d5a19f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3336c8c5cf2e408191449c2a6499c9b4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:30.860247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-98c3078e-f6b6-491c-a62b-bd8ca6d5a19f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3336c8c5cf2e408191449c2a6499c9b4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:30.879565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-98c3078e-f6b6-491c-a62b-bd8ca6d5a19f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3336c8c5cf2e408191449c2a6499c9b4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:30.881786 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-98c3078e-f6b6-491c-a62b-bd8ca6d5a19f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3336c8c5cf2e408191449c2a6499c9b4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:30.901175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 843/1706] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:52:30 2022] DELETE /identity/v3/users/3336c8c5cf2e408191449c2a6499c9b4 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:30.903396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3078e-f6b6-491c-a62b-bd8ca6d5a19f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:30.907906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-98c3078e-f6b6-491c-a62b-bd8ca6d5a19f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:31.028313 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544611f0-0177-40bc-9597-0198c614fcca tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:31.114801 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c3078e-f6b6-491c-a62b-bd8ca6d5a19f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:31.132219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98d703-3320-4dff-924b-5e08153c0bf9 tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbcd9d15f7d2461083b7e005934ad7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26bfcd50a403404c96dec2b695857d26', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:31.133245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98d703-3320-4dff-924b-5e08153c0bf9 tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:31.133546 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98d703-3320-4dff-924b-5e08153c0bf9 tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:31.134472 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98d703-3320-4dff-924b-5e08153c0bf9 tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:31.137423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98d703-3320-4dff-924b-5e08153c0bf9 tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:52:31.137806 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98d703-3320-4dff-924b-5e08153c0bf9 tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:31.146772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98d703-3320-4dff-924b-5e08153c0bf9 tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:31.180024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 844/1707] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:52:31.185892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9f83f-85b6-4d0a-9996-5eed86b30fb7 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:31.188537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9f83f-85b6-4d0a-9996-5eed86b30fb7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:31.188851 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca9f83f-85b6-4d0a-9996-5eed86b30fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:31.248418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb8b7e-758a-4cd4-ba16-933055028a5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:31.249406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6ca9f83f-85b6-4d0a-9996-5eed86b30fb7 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:31.250865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6ca9f83f-85b6-4d0a-9996-5eed86b30fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:31.252497 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb8b7e-758a-4cd4-ba16-933055028a5b service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:31.252839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb8b7e-758a-4cd4-ba16-933055028a5b service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:31.253119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb8b7e-758a-4cd4-ba16-933055028a5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:31.255441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f4eb8b7e-758a-4cd4-ba16-933055028a5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:31.359983 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:31.369535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb8b7e-758a-4cd4-ba16-933055028a5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:31.374658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb8b7e-758a-4cd4-ba16-933055028a5b service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:31.402493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 845/1708] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:31.408216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9f83f-85b6-4d0a-9996-5eed86b30fb7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:31.412045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 864/1709] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:31.430260 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca9f83f-85b6-4d0a-9996-5eed86b30fb7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:31.445479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc11727-0f5b-4a27-999d-f0ac177571db tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bbcd9d15f7d2461083b7e005934ad7c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26bfcd50a403404c96dec2b695857d26', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:31.446146 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc11727-0f5b-4a27-999d-f0ac177571db tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:31.446461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc11727-0f5b-4a27-999d-f0ac177571db tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:31.446738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc11727-0f5b-4a27-999d-f0ac177571db tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] PATH_INFO: `/v3/projects/aefd202f894445ad98a162d953d9ea25` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:31.448075 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc11727-0f5b-4a27-999d-f0ac177571db tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] RBAC: Authorizing `identity:delete_project(project_id=aefd202f894445ad98a162d953d9ea25)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:31.450019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc11727-0f5b-4a27-999d-f0ac177571db tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:31.451712 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:31.476074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb8b7e-758a-4cd4-ba16-933055028a5b service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:31.478730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f4eb8b7e-758a-4cd4-ba16-933055028a5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:31.610964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0fc11727-0f5b-4a27-999d-f0ac177571db tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aefd202f894445ad98a162d953d9ea25', 'request_id': 'req-0fc11727-0f5b-4a27-999d-f0ac177571db'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:31.633395 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0fc11727-0f5b-4a27-999d-f0ac177571db tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aefd202f894445ad98a162d953d9ea25', 'request_id': 'req-0fc11727-0f5b-4a27-999d-f0ac177571db'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:31.649896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0fc11727-0f5b-4a27-999d-f0ac177571db tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aefd202f894445ad98a162d953d9ea25', 'request_id': 'req-0fc11727-0f5b-4a27-999d-f0ac177571db'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:31.670873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb8b7e-758a-4cd4-ba16-933055028a5b service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:31.678931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0fc11727-0f5b-4a27-999d-f0ac177571db tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] The token cache is being invalidate because project aefd202f894445ad98a162d953d9ea25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:52:31.679227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0fc11727-0f5b-4a27-999d-f0ac177571db tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:31.686367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0fc11727-0f5b-4a27-999d-f0ac177571db tempest-VolumeQuotaClassesTest-2111261271 tempest-VolumeQuotaClassesTest-2111261271-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:31.693706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 846/1710] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:52:31 2022] DELETE /identity/v3/projects/aefd202f894445ad98a162d953d9ea25 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:31.856027 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b8ee6-ad02-412f-bff8-299d371cb8a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:31.857695 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b8ee6-ad02-412f-bff8-299d371cb8a8 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:31.857986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b8ee6-ad02-412f-bff8-299d371cb8a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:31.858311 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b8ee6-ad02-412f-bff8-299d371cb8a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:31.863426 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-987b8ee6-ad02-412f-bff8-299d371cb8a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:31.922107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb8aa8-74b4-499d-9a59-dbed93828380 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:31.922542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb8aa8-74b4-499d-9a59-dbed93828380 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:31.922935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb8aa8-74b4-499d-9a59-dbed93828380 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:31.926644 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 847/1711] 10.222.0.60 () {54 vars in 921 bytes} [Mon Sep 19 14:52:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:52:31.981645 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f704b-1ac9-4c17-9eae-56d4ac58a7cc None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:31.981936 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f704b-1ac9-4c17-9eae-56d4ac58a7cc None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:31.982347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f704b-1ac9-4c17-9eae-56d4ac58a7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:32.032671 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9a7f704b-1ac9-4c17-9eae-56d4ac58a7cc None None] MFA Rules not processed for user `568189358b154b66a729fae2f0411d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:32.034135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9a7f704b-1ac9-4c17-9eae-56d4ac58a7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:32.050882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b8ee6-ad02-412f-bff8-299d371cb8a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:32.053095 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b8ee6-ad02-412f-bff8-299d371cb8a8 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:32.070557 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 865/1712] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:52:32.152676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b8ee6-ad02-412f-bff8-299d371cb8a8 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:32.154645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-987b8ee6-ad02-412f-bff8-299d371cb8a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:32.190728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b8ee6-ad02-412f-bff8-299d371cb8a8 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:32.202869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c1b31-a93d-4a9d-ad0c-1bbefe243020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:32.204737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c1b31-a93d-4a9d-ad0c-1bbefe243020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:32.204938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c1b31-a93d-4a9d-ad0c-1bbefe243020 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:32.205111 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c1b31-a93d-4a9d-ad0c-1bbefe243020 admin admin] PATH_INFO: `/v3/users/26cbf33d505641c29f95f20f29a126da` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:32.208942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c1b31-a93d-4a9d-ad0c-1bbefe243020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26cbf33d505641c29f95f20f29a126da)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:32.209601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 848/1713] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Sep 19 14:52:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:52:32.217614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c1b31-a93d-4a9d-ad0c-1bbefe243020 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:32.276295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f704b-1ac9-4c17-9eae-56d4ac58a7cc None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:32.278147 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9a7f704b-1ac9-4c17-9eae-56d4ac58a7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:32.305307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f704b-1ac9-4c17-9eae-56d4ac58a7cc None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:32.325734 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb648b-ccc5-4b42-a68e-acf45fc11829 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '568189358b154b66a729fae2f0411d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:32.326556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb648b-ccc5-4b42-a68e-acf45fc11829 service placement] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:32.328900 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb648b-ccc5-4b42-a68e-acf45fc11829 service placement] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:32.329485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb648b-ccc5-4b42-a68e-acf45fc11829 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:32.342804 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8f7c1b31-a93d-4a9d-ad0c-1bbefe243020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26cbf33d505641c29f95f20f29a126da'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:32.357383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:32.361934 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8f7c1b31-a93d-4a9d-ad0c-1bbefe243020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26cbf33d505641c29f95f20f29a126da'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:32.365469 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb648b-ccc5-4b42-a68e-acf45fc11829 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:32.371422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb648b-ccc5-4b42-a68e-acf45fc11829 service placement] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:32.396309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8f7c1b31-a93d-4a9d-ad0c-1bbefe243020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26cbf33d505641c29f95f20f29a126da'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:32.410772 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8f7c1b31-a93d-4a9d-ad0c-1bbefe243020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26cbf33d505641c29f95f20f29a126da'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:32.411710 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8f7c1b31-a93d-4a9d-ad0c-1bbefe243020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26cbf33d505641c29f95f20f29a126da'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:32.416314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 849/1714] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:52:32.417219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb648b-ccc5-4b42-a68e-acf45fc11829 service placement] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:32.421636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0ecb648b-ccc5-4b42-a68e-acf45fc11829 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:32.425632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 866/1715] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:52:32 2022] DELETE /identity/v3/users/26cbf33d505641c29f95f20f29a126da => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:32.430070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e53ed9-6851-449b-9cd1-b5e9d8c319e6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:32.430359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e53ed9-6851-449b-9cd1-b5e9d8c319e6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:32.430681 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e53ed9-6851-449b-9cd1-b5e9d8c319e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:32.463970 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb648b-ccc5-4b42-a68e-acf45fc11829 service placement] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:32.495423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f6e53ed9-6851-449b-9cd1-b5e9d8c319e6 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:32.500761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f6e53ed9-6851-449b-9cd1-b5e9d8c319e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:32.645896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c91da-5202-48cc-a1d2-35fe2e19391f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:32.646569 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c91da-5202-48cc-a1d2-35fe2e19391f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:32.646868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c91da-5202-48cc-a1d2-35fe2e19391f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:32.647205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46c91da-5202-48cc-a1d2-35fe2e19391f admin admin] PATH_INFO: `/v3/users/ab63f77cf82541feac9f238c301a797a` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:32.651420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c91da-5202-48cc-a1d2-35fe2e19391f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab63f77cf82541feac9f238c301a797a)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:32.659742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46c91da-5202-48cc-a1d2-35fe2e19391f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:32.689397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:32.770065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f46c91da-5202-48cc-a1d2-35fe2e19391f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab63f77cf82541feac9f238c301a797a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:32.801826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 867/1716] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:52:32.804209 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a89bd-2f1a-4afa-85db-affb8457d5d0 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:32.805128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a89bd-2f1a-4afa-85db-affb8457d5d0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:32.805426 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a89bd-2f1a-4afa-85db-affb8457d5d0 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:32.808374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f46c91da-5202-48cc-a1d2-35fe2e19391f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab63f77cf82541feac9f238c301a797a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:32.811999 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 868/1717] 10.222.0.60 () {54 vars in 912 bytes} [Mon Sep 19 14:52:32 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:52:32.817645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a6c68-0966-44f6-95b0-5e1ddbe2da97 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:32.818362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a6c68-0966-44f6-95b0-5e1ddbe2da97 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:32.818773 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a6c68-0966-44f6-95b0-5e1ddbe2da97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:32.882127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f46c91da-5202-48cc-a1d2-35fe2e19391f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab63f77cf82541feac9f238c301a797a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:32.916715 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f46c91da-5202-48cc-a1d2-35fe2e19391f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab63f77cf82541feac9f238c301a797a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:32.917806 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f46c91da-5202-48cc-a1d2-35fe2e19391f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab63f77cf82541feac9f238c301a797a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:32.933440 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 850/1718] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:52:32 2022] DELETE /identity/v3/users/ab63f77cf82541feac9f238c301a797a => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:32.935927 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c91da-5202-48cc-a1d2-35fe2e19391f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:32.940914 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f46c91da-5202-48cc-a1d2-35fe2e19391f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:32.950372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3c8a6c68-0966-44f6-95b0-5e1ddbe2da97 None None] MFA Rules not processed for user `2fbef7b24f134f5eb1193aed1b7ee69c`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:32.965145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3c8a6c68-0966-44f6-95b0-5e1ddbe2da97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:33.120284 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 869/1719] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Sep 19 14:52:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:33.135761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46c91da-5202-48cc-a1d2-35fe2e19391f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:33.151059 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcb609-3f56-4fa0-b24e-a2ee3f0db511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:33.151213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a6c68-0966-44f6-95b0-5e1ddbe2da97 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:33.151811 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcb609-3f56-4fa0-b24e-a2ee3f0db511 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:33.152141 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcb609-3f56-4fa0-b24e-a2ee3f0db511 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:33.152567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3c8a6c68-0966-44f6-95b0-5e1ddbe2da97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:33.153301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcb609-3f56-4fa0-b24e-a2ee3f0db511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:33.155247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-35dcb609-3f56-4fa0-b24e-a2ee3f0db511 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:33.167876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a6c68-0966-44f6-95b0-5e1ddbe2da97 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:33.177337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d83b59-1171-489d-9857-bedfc8013ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:33.181497 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d83b59-1171-489d-9857-bedfc8013ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:33.181788 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d83b59-1171-489d-9857-bedfc8013ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:33.182068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d83b59-1171-489d-9857-bedfc8013ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:33.200969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d83b59-1171-489d-9857-bedfc8013ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:33.202441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d83b59-1171-489d-9857-bedfc8013ca8 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:33.228217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 870/1720] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:33.334833 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcb609-3f56-4fa0-b24e-a2ee3f0db511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:33.340031 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcb609-3f56-4fa0-b24e-a2ee3f0db511 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:33.365279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 851/1721] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:34.159280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d83b59-1171-489d-9857-bedfc8013ca8 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:34.160909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-07d83b59-1171-489d-9857-bedfc8013ca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:34.229303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d83b59-1171-489d-9857-bedfc8013ca8 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:34.251221 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab09842-b021-43d3-952c-dec3e03df7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:34.252100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab09842-b021-43d3-952c-dec3e03df7dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:34.252555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab09842-b021-43d3-952c-dec3e03df7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:34.252971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab09842-b021-43d3-952c-dec3e03df7dd admin admin] PATH_INFO: `/v3/projects/884c136dd0b443f4a0aa1822bf808aab` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:34.255925 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab09842-b021-43d3-952c-dec3e03df7dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=884c136dd0b443f4a0aa1822bf808aab)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:34.258565 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab09842-b021-43d3-952c-dec3e03df7dd admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:34.361297 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4ab09842-b021-43d3-952c-dec3e03df7dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '884c136dd0b443f4a0aa1822bf808aab', 'request_id': 'req-4ab09842-b021-43d3-952c-dec3e03df7dd'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:34.363013 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:34.383166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4ab09842-b021-43d3-952c-dec3e03df7dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '884c136dd0b443f4a0aa1822bf808aab', 'request_id': 'req-4ab09842-b021-43d3-952c-dec3e03df7dd'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:34.394979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4ab09842-b021-43d3-952c-dec3e03df7dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '884c136dd0b443f4a0aa1822bf808aab', 'request_id': 'req-4ab09842-b021-43d3-952c-dec3e03df7dd'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:34.426740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4ab09842-b021-43d3-952c-dec3e03df7dd admin admin] The token cache is being invalidate because project 884c136dd0b443f4a0aa1822bf808aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:52:34.427093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4ab09842-b021-43d3-952c-dec3e03df7dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:34.430064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4ab09842-b021-43d3-952c-dec3e03df7dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:34.436478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 871/1722] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:52:34 2022] DELETE /identity/v3/projects/884c136dd0b443f4a0aa1822bf808aab => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:34.437452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcb609-3f56-4fa0-b24e-a2ee3f0db511 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:34.457859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcb609-3f56-4fa0-b24e-a2ee3f0db511 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:34.476399 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab09842-b021-43d3-952c-dec3e03df7dd admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:34.479238 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4ab09842-b021-43d3-952c-dec3e03df7dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:34.582704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:34.626473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77de6ed-75b9-44d3-b28f-ffcf2e5efbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:34.628015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77de6ed-75b9-44d3-b28f-ffcf2e5efbc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:34.628306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77de6ed-75b9-44d3-b28f-ffcf2e5efbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:34.628713 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77de6ed-75b9-44d3-b28f-ffcf2e5efbc6 admin admin] PATH_INFO: `/v3/users/edd1e7ae073848a49663256c21e35f96` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:34.631083 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77de6ed-75b9-44d3-b28f-ffcf2e5efbc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edd1e7ae073848a49663256c21e35f96)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:34.633131 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77de6ed-75b9-44d3-b28f-ffcf2e5efbc6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:34.660918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab09842-b021-43d3-952c-dec3e03df7dd admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:34.671010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55773af6-e0d8-44d7-b17b-83019821e840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:34.671851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55773af6-e0d8-44d7-b17b-83019821e840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:34.672294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55773af6-e0d8-44d7-b17b-83019821e840 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:34.673813 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55773af6-e0d8-44d7-b17b-83019821e840 admin admin] PATH_INFO: `/v3/projects/09228199bc084a8c89bd14aa0658079a` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:34.675395 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55773af6-e0d8-44d7-b17b-83019821e840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09228199bc084a8c89bd14aa0658079a)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:34.677886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55773af6-e0d8-44d7-b17b-83019821e840 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:34.732414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f77de6ed-75b9-44d3-b28f-ffcf2e5efbc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edd1e7ae073848a49663256c21e35f96'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:34.757380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f77de6ed-75b9-44d3-b28f-ffcf2e5efbc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edd1e7ae073848a49663256c21e35f96'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:34.784935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f77de6ed-75b9-44d3-b28f-ffcf2e5efbc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edd1e7ae073848a49663256c21e35f96'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:34.797635 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f77de6ed-75b9-44d3-b28f-ffcf2e5efbc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edd1e7ae073848a49663256c21e35f96'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:34.799631 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f77de6ed-75b9-44d3-b28f-ffcf2e5efbc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edd1e7ae073848a49663256c21e35f96'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:34.816567 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 852/1723] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:52:34 2022] DELETE /identity/v3/users/edd1e7ae073848a49663256c21e35f96 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:34.827063 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-55773af6-e0d8-44d7-b17b-83019821e840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09228199bc084a8c89bd14aa0658079a', 'request_id': 'req-55773af6-e0d8-44d7-b17b-83019821e840'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:34.839358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-55773af6-e0d8-44d7-b17b-83019821e840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09228199bc084a8c89bd14aa0658079a', 'request_id': 'req-55773af6-e0d8-44d7-b17b-83019821e840'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:34.847793 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-55773af6-e0d8-44d7-b17b-83019821e840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09228199bc084a8c89bd14aa0658079a', 'request_id': 'req-55773af6-e0d8-44d7-b17b-83019821e840'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:34.869766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-55773af6-e0d8-44d7-b17b-83019821e840 admin admin] The token cache is being invalidate because project 09228199bc084a8c89bd14aa0658079a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:52:34.870582 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-55773af6-e0d8-44d7-b17b-83019821e840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:34.871445 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-55773af6-e0d8-44d7-b17b-83019821e840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:34.881295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 872/1724] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:52:34 2022] DELETE /identity/v3/projects/09228199bc084a8c89bd14aa0658079a => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:52:34.887581 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77de6ed-75b9-44d3-b28f-ffcf2e5efbc6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:34.890446 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f77de6ed-75b9-44d3-b28f-ffcf2e5efbc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:34.969224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55773af6-e0d8-44d7-b17b-83019821e840 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:34.972926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-55773af6-e0d8-44d7-b17b-83019821e840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:35.016287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77de6ed-75b9-44d3-b28f-ffcf2e5efbc6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:35.049781 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7f0c5-9d28-4765-b3a8-551d2068b736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:35.051588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7f0c5-9d28-4765-b3a8-551d2068b736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:35.051588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7f0c5-9d28-4765-b3a8-551d2068b736 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:35.053777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7f0c5-9d28-4765-b3a8-551d2068b736 admin admin] PATH_INFO: `/v3/users/bbcd9d15f7d2461083b7e005934ad7c1` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:35.058313 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7f0c5-9d28-4765-b3a8-551d2068b736 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbcd9d15f7d2461083b7e005934ad7c1)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:35.061774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7f0c5-9d28-4765-b3a8-551d2068b736 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:35.174295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55773af6-e0d8-44d7-b17b-83019821e840 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:35.182496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7be7f0c5-9d28-4765-b3a8-551d2068b736 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbcd9d15f7d2461083b7e005934ad7c1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:35.188009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82735f68-2196-4236-aa7d-03bab877d115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:35.189004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82735f68-2196-4236-aa7d-03bab877d115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:35.190502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82735f68-2196-4236-aa7d-03bab877d115 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:35.199583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82735f68-2196-4236-aa7d-03bab877d115 admin admin] PATH_INFO: `/v3/projects/c75feb7246f543b2aab610022f5f1d8d` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:35.205387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82735f68-2196-4236-aa7d-03bab877d115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c75feb7246f543b2aab610022f5f1d8d)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:35.212793 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82735f68-2196-4236-aa7d-03bab877d115 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:35.213505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7be7f0c5-9d28-4765-b3a8-551d2068b736 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbcd9d15f7d2461083b7e005934ad7c1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:35.231347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:35.249476 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:35.259306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7be7f0c5-9d28-4765-b3a8-551d2068b736 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbcd9d15f7d2461083b7e005934ad7c1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:35.278628 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7be7f0c5-9d28-4765-b3a8-551d2068b736 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbcd9d15f7d2461083b7e005934ad7c1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:35.279753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7be7f0c5-9d28-4765-b3a8-551d2068b736 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbcd9d15f7d2461083b7e005934ad7c1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:35.308336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 853/1725] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:52:34 2022] DELETE /identity/v3/users/bbcd9d15f7d2461083b7e005934ad7c1 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:35.369938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-82735f68-2196-4236-aa7d-03bab877d115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c75feb7246f543b2aab610022f5f1d8d', 'request_id': 'req-82735f68-2196-4236-aa7d-03bab877d115'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:35.385173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-82735f68-2196-4236-aa7d-03bab877d115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c75feb7246f543b2aab610022f5f1d8d', 'request_id': 'req-82735f68-2196-4236-aa7d-03bab877d115'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:35.417208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-82735f68-2196-4236-aa7d-03bab877d115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c75feb7246f543b2aab610022f5f1d8d', 'request_id': 'req-82735f68-2196-4236-aa7d-03bab877d115'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:35.457701 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-82735f68-2196-4236-aa7d-03bab877d115 admin admin] The token cache is being invalidate because project c75feb7246f543b2aab610022f5f1d8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:52:35.457994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-82735f68-2196-4236-aa7d-03bab877d115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:35.459172 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-82735f68-2196-4236-aa7d-03bab877d115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:35.462627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 873/1726] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:52:34 2022] DELETE /identity/v3/projects/c75feb7246f543b2aab610022f5f1d8d => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:35.467566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7f0c5-9d28-4765-b3a8-551d2068b736 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:35.470394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7be7f0c5-9d28-4765-b3a8-551d2068b736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:35.816616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7f0c5-9d28-4765-b3a8-551d2068b736 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:35.932139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82735f68-2196-4236-aa7d-03bab877d115 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:35.934881 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-82735f68-2196-4236-aa7d-03bab877d115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:35.969194 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52f775-8bdd-4c91-878d-4956bf68f5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:35.970481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52f775-8bdd-4c91-878d-4956bf68f5c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:35.971108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52f775-8bdd-4c91-878d-4956bf68f5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:35.971554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52f775-8bdd-4c91-878d-4956bf68f5c6 admin admin] PATH_INFO: `/v3/users/cd20577980ca46c5bb74879061aee21f` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:35.974405 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc52f775-8bdd-4c91-878d-4956bf68f5c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd20577980ca46c5bb74879061aee21f)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:35.978054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc52f775-8bdd-4c91-878d-4956bf68f5c6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:36.109845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc52f775-8bdd-4c91-878d-4956bf68f5c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd20577980ca46c5bb74879061aee21f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:36.131076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc52f775-8bdd-4c91-878d-4956bf68f5c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd20577980ca46c5bb74879061aee21f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:36.157694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc52f775-8bdd-4c91-878d-4956bf68f5c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd20577980ca46c5bb74879061aee21f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:36.161365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82735f68-2196-4236-aa7d-03bab877d115 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:36.175452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf66a60-53a2-4af5-8e0f-7b7526208a43 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '568189358b154b66a729fae2f0411d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:36.177329 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf66a60-53a2-4af5-8e0f-7b7526208a43 service placement] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:36.186322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf66a60-53a2-4af5-8e0f-7b7526208a43 service placement] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:36.186322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf66a60-53a2-4af5-8e0f-7b7526208a43 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:36.192370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc52f775-8bdd-4c91-878d-4956bf68f5c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd20577980ca46c5bb74879061aee21f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:36.194199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc52f775-8bdd-4c91-878d-4956bf68f5c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd20577980ca46c5bb74879061aee21f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:36.195678 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2bf66a60-53a2-4af5-8e0f-7b7526208a43 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:36.205459 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 854/1727] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:52:35 2022] DELETE /identity/v3/users/cd20577980ca46c5bb74879061aee21f => generated 0 bytes in 736 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:36.205459 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52f775-8bdd-4c91-878d-4956bf68f5c6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:36.206714 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bc52f775-8bdd-4c91-878d-4956bf68f5c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:36.231974 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf66a60-53a2-4af5-8e0f-7b7526208a43 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:36.233607 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf66a60-53a2-4af5-8e0f-7b7526208a43 service placement] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:36.237266 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:36.280721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 874/1728] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:52:36.282253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414e9be-1917-442f-8ac6-3f2b8299fd2b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:36.282544 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414e9be-1917-442f-8ac6-3f2b8299fd2b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:36.282822 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414e9be-1917-442f-8ac6-3f2b8299fd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:36.353917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b414e9be-1917-442f-8ac6-3f2b8299fd2b None None] MFA Rules not processed for user `1cccf354894c4311b7710d8a65682eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:36.356994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b414e9be-1917-442f-8ac6-3f2b8299fd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:36.368848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52f775-8bdd-4c91-878d-4956bf68f5c6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:36.392446 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e1066-67a8-4e76-b036-3a5df1efb5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:36.393116 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e1066-67a8-4e76-b036-3a5df1efb5f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:36.393395 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e1066-67a8-4e76-b036-3a5df1efb5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:36.393833 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e1066-67a8-4e76-b036-3a5df1efb5f5 admin admin] PATH_INFO: `/v3/projects/33209bce29e74c68aae29cbb072d0d01` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:36.395951 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e1066-67a8-4e76-b036-3a5df1efb5f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33209bce29e74c68aae29cbb072d0d01)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:36.402663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e1066-67a8-4e76-b036-3a5df1efb5f5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:36.588967 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 875/1729] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:36.590410 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385923a-caab-474e-9444-98348103c412 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:36.590721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385923a-caab-474e-9444-98348103c412 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:36.591001 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385923a-caab-474e-9444-98348103c412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:36.606228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d30e1066-67a8-4e76-b036-3a5df1efb5f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33209bce29e74c68aae29cbb072d0d01', 'request_id': 'req-d30e1066-67a8-4e76-b036-3a5df1efb5f5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:36.616667 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d30e1066-67a8-4e76-b036-3a5df1efb5f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33209bce29e74c68aae29cbb072d0d01', 'request_id': 'req-d30e1066-67a8-4e76-b036-3a5df1efb5f5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:36.645174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d30e1066-67a8-4e76-b036-3a5df1efb5f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33209bce29e74c68aae29cbb072d0d01', 'request_id': 'req-d30e1066-67a8-4e76-b036-3a5df1efb5f5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:36.679615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3385923a-caab-474e-9444-98348103c412 None None] MFA Rules not processed for user `b2c40600d2fd4eea8e4ba999050abd73`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:36.687104 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d30e1066-67a8-4e76-b036-3a5df1efb5f5 admin admin] The token cache is being invalidate because project 33209bce29e74c68aae29cbb072d0d01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:52:36.689522 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3385923a-caab-474e-9444-98348103c412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:36.690007 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d30e1066-67a8-4e76-b036-3a5df1efb5f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:36.691083 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d30e1066-67a8-4e76-b036-3a5df1efb5f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:36.695569 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 855/1730] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:52:36 2022] DELETE /identity/v3/projects/33209bce29e74c68aae29cbb072d0d01 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:36.700810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a09245-d469-4e4a-96b6-1e9220255fda None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:36.701205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a09245-d469-4e4a-96b6-1e9220255fda None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:36.701692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a09245-d469-4e4a-96b6-1e9220255fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:36.768763 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e7a09245-d469-4e4a-96b6-1e9220255fda None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:36.771717 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e7a09245-d469-4e4a-96b6-1e9220255fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:36.793880 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:36.930123 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 876/1731] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:36.936824 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385923a-caab-474e-9444-98348103c412 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:36.939307 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3385923a-caab-474e-9444-98348103c412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:37.008415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 856/1732] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:37.010609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d093b6f-3984-4e52-bf81-99691463c50b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:37.010997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d093b6f-3984-4e52-bf81-99691463c50b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:37.011148 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d093b6f-3984-4e52-bf81-99691463c50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:37.065852 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8d093b6f-3984-4e52-bf81-99691463c50b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:37.073113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8d093b6f-3984-4e52-bf81-99691463c50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:37.130066 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 857/1733] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:37.135017 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d093b6f-3984-4e52-bf81-99691463c50b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:37.164726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385923a-caab-474e-9444-98348103c412 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:37.168399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d093b6f-3984-4e52-bf81-99691463c50b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:37.179787 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205ec85-3aa7-4a1f-93b0-9de825ab23b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:37.180919 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205ec85-3aa7-4a1f-93b0-9de825ab23b6 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:37.181883 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205ec85-3aa7-4a1f-93b0-9de825ab23b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:37.183870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205ec85-3aa7-4a1f-93b0-9de825ab23b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:37.187218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326468fe-8645-48e7-9d43-3f24f0d45391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:37.188809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205ec85-3aa7-4a1f-93b0-9de825ab23b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:52:37.189188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205ec85-3aa7-4a1f-93b0-9de825ab23b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:37.190853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8205ec85-3aa7-4a1f-93b0-9de825ab23b6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:37.191166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326468fe-8645-48e7-9d43-3f24f0d45391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:37.191387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326468fe-8645-48e7-9d43-3f24f0d45391 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:37.192585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326468fe-8645-48e7-9d43-3f24f0d45391 admin admin] PATH_INFO: `/v3/projects/e5676c2d31e7404eb0045d67130563ae` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:37.197690 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326468fe-8645-48e7-9d43-3f24f0d45391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5676c2d31e7404eb0045d67130563ae)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:37.202054 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326468fe-8645-48e7-9d43-3f24f0d45391 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:37.245232 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 858/1734] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:52:37.253202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205ec85-3aa7-4a1f-93b0-9de825ab23b6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:37.273228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-326468fe-8645-48e7-9d43-3f24f0d45391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5676c2d31e7404eb0045d67130563ae', 'request_id': 'req-326468fe-8645-48e7-9d43-3f24f0d45391'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:37.284971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-326468fe-8645-48e7-9d43-3f24f0d45391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5676c2d31e7404eb0045d67130563ae', 'request_id': 'req-326468fe-8645-48e7-9d43-3f24f0d45391'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:37.287119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205ec85-3aa7-4a1f-93b0-9de825ab23b6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:37.294794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-326468fe-8645-48e7-9d43-3f24f0d45391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5676c2d31e7404eb0045d67130563ae', 'request_id': 'req-326468fe-8645-48e7-9d43-3f24f0d45391'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:37.316809 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-326468fe-8645-48e7-9d43-3f24f0d45391 admin admin] The token cache is being invalidate because project e5676c2d31e7404eb0045d67130563ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:52:37.317273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-326468fe-8645-48e7-9d43-3f24f0d45391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:37.317961 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-326468fe-8645-48e7-9d43-3f24f0d45391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:37.320416 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 877/1735] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:52:36 2022] DELETE /identity/v3/projects/e5676c2d31e7404eb0045d67130563ae => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:37.330494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0279ffc-aa47-4fb5-aee8-994ffa7b9eea None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:37.331895 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0279ffc-aa47-4fb5-aee8-994ffa7b9eea None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:37.333682 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:37.340096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0279ffc-aa47-4fb5-aee8-994ffa7b9eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:37.398096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d0279ffc-aa47-4fb5-aee8-994ffa7b9eea None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:37.400255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d0279ffc-aa47-4fb5-aee8-994ffa7b9eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:37.441841 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2906e3-6413-4d30-b643-09473b57af0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:37.442734 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2906e3-6413-4d30-b643-09473b57af0e admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:37.443030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2906e3-6413-4d30-b643-09473b57af0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:37.443305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2906e3-6413-4d30-b643-09473b57af0e admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:37.443984 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2906e3-6413-4d30-b643-09473b57af0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:37.447716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2906e3-6413-4d30-b643-09473b57af0e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:37.505591 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 859/1736] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:37 2022] POST /identity/v3/projects => generated 355 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:37.508204 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2906e3-6413-4d30-b643-09473b57af0e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:37.525509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dd2906e3-6413-4d30-b643-09473b57af0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:37.549774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 878/1737] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:37.549774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0279ffc-aa47-4fb5-aee8-994ffa7b9eea None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:37.552739 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d0279ffc-aa47-4fb5-aee8-994ffa7b9eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:37.604941 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0279ffc-aa47-4fb5-aee8-994ffa7b9eea None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:37.624525 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:37.690195 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2906e3-6413-4d30-b643-09473b57af0e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:37.702217 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442de5dd-1ab3-47b0-933d-15145929da1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:37.704256 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442de5dd-1ab3-47b0-933d-15145929da1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:37.704605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442de5dd-1ab3-47b0-933d-15145929da1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:37.704903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442de5dd-1ab3-47b0-933d-15145929da1a admin admin] PATH_INFO: `/v3/projects/26bfcd50a403404c96dec2b695857d26` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:37.706251 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442de5dd-1ab3-47b0-933d-15145929da1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=26bfcd50a403404c96dec2b695857d26)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:37.712893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442de5dd-1ab3-47b0-933d-15145929da1a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:37.764996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da3e56-80c8-4f89-9e0d-c5ff2f3f0d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:37.769016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da3e56-80c8-4f89-9e0d-c5ff2f3f0d1c admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:37.769016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da3e56-80c8-4f89-9e0d-c5ff2f3f0d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:37.769016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da3e56-80c8-4f89-9e0d-c5ff2f3f0d1c admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:37.770534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da3e56-80c8-4f89-9e0d-c5ff2f3f0d1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:37.776524 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da3e56-80c8-4f89-9e0d-c5ff2f3f0d1c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:37.841231 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 879/1738] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:37 2022] POST /identity/v3/users => generated 278 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:37.842641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a04ad4-294c-40d8-945e-047851da098c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:37.842954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a04ad4-294c-40d8-945e-047851da098c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:37.843210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a04ad4-294c-40d8-945e-047851da098c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:37.846219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-442de5dd-1ab3-47b0-933d-15145929da1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26bfcd50a403404c96dec2b695857d26', 'request_id': 'req-442de5dd-1ab3-47b0-933d-15145929da1a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:37.876467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-442de5dd-1ab3-47b0-933d-15145929da1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26bfcd50a403404c96dec2b695857d26', 'request_id': 'req-442de5dd-1ab3-47b0-933d-15145929da1a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:37.906706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-442de5dd-1ab3-47b0-933d-15145929da1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26bfcd50a403404c96dec2b695857d26', 'request_id': 'req-442de5dd-1ab3-47b0-933d-15145929da1a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:37.935530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-21a04ad4-294c-40d8-945e-047851da098c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:37.938002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-442de5dd-1ab3-47b0-933d-15145929da1a admin admin] The token cache is being invalidate because project 26bfcd50a403404c96dec2b695857d26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:52:37.938988 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-442de5dd-1ab3-47b0-933d-15145929da1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:37.939405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-21a04ad4-294c-40d8-945e-047851da098c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:37.940249 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-442de5dd-1ab3-47b0-933d-15145929da1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:37.944422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 860/1739] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:52:37 2022] DELETE /identity/v3/projects/26bfcd50a403404c96dec2b695857d26 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:37.947872 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442de5dd-1ab3-47b0-933d-15145929da1a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:37.950755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-442de5dd-1ab3-47b0-933d-15145929da1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:38.057127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:38.124461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 880/1740] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:38.126022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f17411-bda9-4d04-b08f-48feb2b695e5 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:38.126383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f17411-bda9-4d04-b08f-48feb2b695e5 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:38.126737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f17411-bda9-4d04-b08f-48feb2b695e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:38.168575 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442de5dd-1ab3-47b0-933d-15145929da1a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:38.184857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432bd0c-2ffb-4211-b538-ff773efb8fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:38.185810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432bd0c-2ffb-4211-b538-ff773efb8fee admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:38.186346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432bd0c-2ffb-4211-b538-ff773efb8fee admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:38.186801 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432bd0c-2ffb-4211-b538-ff773efb8fee admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:38.187644 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432bd0c-2ffb-4211-b538-ff773efb8fee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:38.189625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1432bd0c-2ffb-4211-b538-ff773efb8fee admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:38.190877 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b2f17411-bda9-4d04-b08f-48feb2b695e5 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:38.195746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b2f17411-bda9-4d04-b08f-48feb2b695e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:38.215635 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 861/1741] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:38.219647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b5fc-c544-4227-871f-21c24d1a3421 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:38.220179 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b5fc-c544-4227-871f-21c24d1a3421 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:38.222901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b5fc-c544-4227-871f-21c24d1a3421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:38.223187 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 881/1742] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:38.225772 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f17411-bda9-4d04-b08f-48feb2b695e5 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:38.250618 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f17411-bda9-4d04-b08f-48feb2b695e5 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:38.263860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfc07c-38d0-4d9d-9da8-a227ad2572fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:38.264498 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfc07c-38d0-4d9d-9da8-a227ad2572fd admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:38.264775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfc07c-38d0-4d9d-9da8-a227ad2572fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:38.265044 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfc07c-38d0-4d9d-9da8-a227ad2572fd admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:38.265687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfc07c-38d0-4d9d-9da8-a227ad2572fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:52:38.265986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfc07c-38d0-4d9d-9da8-a227ad2572fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:38.267744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfc07c-38d0-4d9d-9da8-a227ad2572fd admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:38.297692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 882/1743] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:52:38.299433 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2fc64-c322-417a-acf9-003a1a8870a7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:38.299742 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2fc64-c322-417a-acf9-003a1a8870a7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:38.300038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2fc64-c322-417a-acf9-003a1a8870a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:38.302914 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d790b5fc-c544-4227-871f-21c24d1a3421 None None] MFA Rules not processed for user `61743a63def7451d9b739f3b17afe8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:38.308317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d790b5fc-c544-4227-871f-21c24d1a3421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:38.341067 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9fd2fc64-c322-417a-acf9-003a1a8870a7 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:38.345625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9fd2fc64-c322-417a-acf9-003a1a8870a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:38.370568 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 883/1744] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:38.371536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2fc64-c322-417a-acf9-003a1a8870a7 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:38.391488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2fc64-c322-417a-acf9-003a1a8870a7 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:38.405636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ac534-8292-4bf9-a7c0-094ea0b01d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:38.406258 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ac534-8292-4bf9-a7c0-094ea0b01d65 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:38.406540 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ac534-8292-4bf9-a7c0-094ea0b01d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:38.406809 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ac534-8292-4bf9-a7c0-094ea0b01d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:38.407447 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ac534-8292-4bf9-a7c0-094ea0b01d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:38.413705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ac534-8292-4bf9-a7c0-094ea0b01d65 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:38.441421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 884/1745] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:38 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:38.444593 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ac534-8292-4bf9-a7c0-094ea0b01d65 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:38.478038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ac534-8292-4bf9-a7c0-094ea0b01d65 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:38.490399 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f3def-3a50-4a60-bfdf-db804121e2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:38.491311 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f3def-3a50-4a60-bfdf-db804121e2c1 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:38.493791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f3def-3a50-4a60-bfdf-db804121e2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:38.494177 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f3def-3a50-4a60-bfdf-db804121e2c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:38.494942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f3def-3a50-4a60-bfdf-db804121e2c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:38.499136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f3def-3a50-4a60-bfdf-db804121e2c1 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:38.522240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 862/1746] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:38.523221 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790b5fc-c544-4227-871f-21c24d1a3421 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:38.547551 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790b5fc-c544-4227-871f-21c24d1a3421 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:38.565983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e77c1-9a6e-4d6c-a225-fb4a51c96c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:38.568006 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e77c1-9a6e-4d6c-a225-fb4a51c96c62 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:38.570286 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e77c1-9a6e-4d6c-a225-fb4a51c96c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:38.570794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e77c1-9a6e-4d6c-a225-fb4a51c96c62 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:38.571677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e77c1-9a6e-4d6c-a225-fb4a51c96c62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:52:38.573149 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 885/1747] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:38 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:38.574242 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f3def-3a50-4a60-bfdf-db804121e2c1 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:38.575010 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e77c1-9a6e-4d6c-a225-fb4a51c96c62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:38.581015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35e77c1-9a6e-4d6c-a225-fb4a51c96c62 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:38.624289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 863/1748] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:52:38.625836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e77c1-9a6e-4d6c-a225-fb4a51c96c62 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:38.661195 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e77c1-9a6e-4d6c-a225-fb4a51c96c62 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:38.764896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:38.878179 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc02bc4-bee3-46a1-9d11-b8d7ade294f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:38.878975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc02bc4-bee3-46a1-9d11-b8d7ade294f2 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:38.879348 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc02bc4-bee3-46a1-9d11-b8d7ade294f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:38.879758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc02bc4-bee3-46a1-9d11-b8d7ade294f2 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:38.880627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc02bc4-bee3-46a1-9d11-b8d7ade294f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:38.882859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc02bc4-bee3-46a1-9d11-b8d7ade294f2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:38.947933 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 864/1749] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:38 2022] POST /identity/v3/users => generated 289 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:38.949210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc02bc4-bee3-46a1-9d11-b8d7ade294f2 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:38.964110 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f3def-3a50-4a60-bfdf-db804121e2c1 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:38.981792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc02bc4-bee3-46a1-9d11-b8d7ade294f2 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:38.989512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008fcec-b562-4ab5-b38c-9a8730823c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:38.990212 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008fcec-b562-4ab5-b38c-9a8730823c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:38.991914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008fcec-b562-4ab5-b38c-9a8730823c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:38.992216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008fcec-b562-4ab5-b38c-9a8730823c33 admin admin] PATH_INFO: `/v3/projects/8e4e7b1217614c50a78d547810a6b238/users/17d659700c0245e1929496c71b0635d6/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:38.997437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e8a4c-31b0-4037-9c3f-368fa83763de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:38.998208 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e8a4c-31b0-4037-9c3f-368fa83763de admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:38.998586 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e8a4c-31b0-4037-9c3f-368fa83763de admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:38.998956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e8a4c-31b0-4037-9c3f-368fa83763de admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:38.999775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e8a4c-31b0-4037-9c3f-368fa83763de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:39.003085 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e8a4c-31b0-4037-9c3f-368fa83763de admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:39.069740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008fcec-b562-4ab5-b38c-9a8730823c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e4e7b1217614c50a78d547810a6b238, user_id=17d659700c0245e1929496c71b0635d6, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:39.084893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008fcec-b562-4ab5-b38c-9a8730823c33 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:39.107266 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 865/1750] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:38 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:39.111798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fb363-244b-491b-b41f-c2810270c9f6 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:39.116090 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 886/1751] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:38 2022] PUT /identity/v3/projects/8e4e7b1217614c50a78d547810a6b238/users/17d659700c0245e1929496c71b0635d6/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:39.116998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008fcec-b562-4ab5-b38c-9a8730823c33 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:39.121704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fb363-244b-491b-b41f-c2810270c9f6 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:39.123588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210fb363-244b-491b-b41f-c2810270c9f6 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:39.135928 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 866/1752] 10.222.0.60 () {54 vars in 912 bytes} [Mon Sep 19 14:52:39 2022] GET /identity => generated 270 bytes in 28 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:52:39.136919 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fb363-244b-491b-b41f-c2810270c9f6 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:39.143977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-210fb363-244b-491b-b41f-c2810270c9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:39.204346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:39.423562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008fcec-b562-4ab5-b38c-9a8730823c33 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:39.451548 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0dd464-c689-4a4c-88d3-4ea0d1af4bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:39.452592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210fb363-244b-491b-b41f-c2810270c9f6 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:39.453975 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0dd464-c689-4a4c-88d3-4ea0d1af4bef admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:39.454810 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0dd464-c689-4a4c-88d3-4ea0d1af4bef admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:39.455090 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0dd464-c689-4a4c-88d3-4ea0d1af4bef admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:39.455755 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0dd464-c689-4a4c-88d3-4ea0d1af4bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:39.457611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0dd464-c689-4a4c-88d3-4ea0d1af4bef admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:39.473535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd1b10-2103-4e77-bd15-2ec2e1b73a71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:39.473967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd1b10-2103-4e77-bd15-2ec2e1b73a71 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:39.474301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd1b10-2103-4e77-bd15-2ec2e1b73a71 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:39.474622 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd1b10-2103-4e77-bd15-2ec2e1b73a71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:39.496224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 887/1753] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:39.497100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0dd464-c689-4a4c-88d3-4ea0d1af4bef admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:39.548227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0dd464-c689-4a4c-88d3-4ea0d1af4bef admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:39.564493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854a4ca-bafb-4d8a-99e6-628d23668198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:39.565398 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854a4ca-bafb-4d8a-99e6-628d23668198 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:39.565836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854a4ca-bafb-4d8a-99e6-628d23668198 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:39.566410 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854a4ca-bafb-4d8a-99e6-628d23668198 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:39.568382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854a4ca-bafb-4d8a-99e6-628d23668198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:39.570422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854a4ca-bafb-4d8a-99e6-628d23668198 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:39.625227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 888/1754] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:39 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:39.626748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf09db0-b3f5-4f79-9edf-84b828b2dd30 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:39.627037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf09db0-b3f5-4f79-9edf-84b828b2dd30 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:39.628829 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf09db0-b3f5-4f79-9edf-84b828b2dd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:39.648427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd1b10-2103-4e77-bd15-2ec2e1b73a71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:39.650142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd1b10-2103-4e77-bd15-2ec2e1b73a71 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:39.701811 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 867/1755] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:39.703292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f4bba-b90f-4331-b147-72bf80b750e2 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:39.703592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f4bba-b90f-4331-b147-72bf80b750e2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:39.705016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f4bba-b90f-4331-b147-72bf80b750e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:39.735238 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3cf09db0-b3f5-4f79-9edf-84b828b2dd30 None None] MFA Rules not processed for user `17d659700c0245e1929496c71b0635d6`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:39.738911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3cf09db0-b3f5-4f79-9edf-84b828b2dd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:39.778912 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1c2f4bba-b90f-4331-b147-72bf80b750e2 None None] MFA Rules not processed for user `2fbef7b24f134f5eb1193aed1b7ee69c`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:39.783783 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1c2f4bba-b90f-4331-b147-72bf80b750e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:39.885834 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 889/1756] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:39.886804 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf09db0-b3f5-4f79-9edf-84b828b2dd30 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:39.905333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf09db0-b3f5-4f79-9edf-84b828b2dd30 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:39.919181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba81fd-9f0d-4bb3-97ca-7894641ff3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:39.920108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba81fd-9f0d-4bb3-97ca-7894641ff3d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:39.920753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba81fd-9f0d-4bb3-97ca-7894641ff3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:39.921265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba81fd-9f0d-4bb3-97ca-7894641ff3d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:39.922327 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba81fd-9f0d-4bb3-97ca-7894641ff3d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:39.924866 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddba81fd-9f0d-4bb3-97ca-7894641ff3d4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:39.931395 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 868/1757] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Sep 19 14:52:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:39.938454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f4bba-b90f-4331-b147-72bf80b750e2 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:39.944345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 890/1758] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:39.944345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebbc4-0c80-453c-8a27-915fe42f2c9a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:39.944345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebbc4-0c80-453c-8a27-915fe42f2c9a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:39.944345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebbc4-0c80-453c-8a27-915fe42f2c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:39.967231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f4bba-b90f-4331-b147-72bf80b750e2 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:39.986095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c8288-ac67-409d-8d73-94719c4e5bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:39.987379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c8288-ac67-409d-8d73-94719c4e5bba admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:39.987604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c8288-ac67-409d-8d73-94719c4e5bba admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:39.988096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c8288-ac67-409d-8d73-94719c4e5bba admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:39.988849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c8288-ac67-409d-8d73-94719c4e5bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:39.990565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c8288-ac67-409d-8d73-94719c4e5bba admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:39.993014 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-fadebbc4-0c80-453c-8a27-915fe42f2c9a None None] MFA Rules not processed for user `49b775647ffb4500911c8ec8cf2ec804`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:39.994594 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fadebbc4-0c80-453c-8a27-915fe42f2c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:40.006089 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:40.022730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 869/1759] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:40.024901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c8288-ac67-409d-8d73-94719c4e5bba admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:40.045914 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c8288-ac67-409d-8d73-94719c4e5bba admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:40.076911 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dcbe7-da6d-4b0d-a059-6c00ec8f0b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:40.078163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dcbe7-da6d-4b0d-a059-6c00ec8f0b03 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:40.079537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dcbe7-da6d-4b0d-a059-6c00ec8f0b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:40.081038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dcbe7-da6d-4b0d-a059-6c00ec8f0b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:40.081735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53dcbe7-da6d-4b0d-a059-6c00ec8f0b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:40.088224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53dcbe7-da6d-4b0d-a059-6c00ec8f0b03 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:40.162319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 870/1760] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:40 2022] POST /identity/v3/projects => generated 357 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:40.163308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dcbe7-da6d-4b0d-a059-6c00ec8f0b03 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:40.169844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 891/1761] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:40.170707 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadebbc4-0c80-453c-8a27-915fe42f2c9a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:40.172052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fadebbc4-0c80-453c-8a27-915fe42f2c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:40.193378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadebbc4-0c80-453c-8a27-915fe42f2c9a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:40.240787 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:40.317938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dcbe7-da6d-4b0d-a059-6c00ec8f0b03 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:40.319338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3424d-75dc-4a86-964d-3d72d5632b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:40.321667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3424d-75dc-4a86-964d-3d72d5632b0d service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:40.323010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3424d-75dc-4a86-964d-3d72d5632b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:40.323310 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3424d-75dc-4a86-964d-3d72d5632b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:40.324329 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d96eb9-22e1-4944-8aad-58cf14f0be42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:40.325278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d96eb9-22e1-4944-8aad-58cf14f0be42 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:40.325707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d96eb9-22e1-4944-8aad-58cf14f0be42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:40.326127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d96eb9-22e1-4944-8aad-58cf14f0be42 admin admin] PATH_INFO: `/v3/projects/a2d404cba14e4bcf8fbcf1d6d622d22e/users/5f09111d61c4498eb51180fb08e19772/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:40.345307 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3424d-75dc-4a86-964d-3d72d5632b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:40.349220 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d96eb9-22e1-4944-8aad-58cf14f0be42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d404cba14e4bcf8fbcf1d6d622d22e, user_id=5f09111d61c4498eb51180fb08e19772, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:40.349374 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3424d-75dc-4a86-964d-3d72d5632b0d service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:40.353258 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d96eb9-22e1-4944-8aad-58cf14f0be42 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:40.367558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 892/1762] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:40.368551 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3424d-75dc-4a86-964d-3d72d5632b0d service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:40.382434 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3424d-75dc-4a86-964d-3d72d5632b0d service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:40.397259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 871/1763] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:40 2022] PUT /identity/v3/projects/a2d404cba14e4bcf8fbcf1d6d622d22e/users/5f09111d61c4498eb51180fb08e19772/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:40.398101 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d96eb9-22e1-4944-8aad-58cf14f0be42 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:40.462630 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9442af3-cb66-4e3e-ab9c-e8ed2e48b89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:40.463746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9442af3-cb66-4e3e-ab9c-e8ed2e48b89c admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:40.463746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9442af3-cb66-4e3e-ab9c-e8ed2e48b89c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:40.463921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9442af3-cb66-4e3e-ab9c-e8ed2e48b89c admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:40.464485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9442af3-cb66-4e3e-ab9c-e8ed2e48b89c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:40.466897 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9442af3-cb66-4e3e-ab9c-e8ed2e48b89c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:40.485641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 893/1764] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:40.485641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b517c-7b67-4f24-bf37-3a9e9a73b114 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:40.485641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b517c-7b67-4f24-bf37-3a9e9a73b114 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:40.485641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b517c-7b67-4f24-bf37-3a9e9a73b114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:40.508656 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-890b517c-7b67-4f24-bf37-3a9e9a73b114 None None] MFA Rules not processed for user `49b775647ffb4500911c8ec8cf2ec804`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:40.509910 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-890b517c-7b67-4f24-bf37-3a9e9a73b114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:40.613747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d96eb9-22e1-4944-8aad-58cf14f0be42 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:40.619654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:40.623475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6777aa-4abe-44ef-bc4b-4234c56fb5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:40.624144 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6777aa-4abe-44ef-bc4b-4234c56fb5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:40.624427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6777aa-4abe-44ef-bc4b-4234c56fb5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:40.624701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6777aa-4abe-44ef-bc4b-4234c56fb5b1 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:40.625443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6777aa-4abe-44ef-bc4b-4234c56fb5b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:40.627289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6777aa-4abe-44ef-bc4b-4234c56fb5b1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:40.660078 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 894/1765] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:40.674657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 872/1766] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:40 2022] POST /identity/v3/users => generated 285 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:40.675701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6777aa-4abe-44ef-bc4b-4234c56fb5b1 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:40.680731 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362621e6-510a-489a-a940-9dd64ae2fb03 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:40.681339 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362621e6-510a-489a-a940-9dd64ae2fb03 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:40.681676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362621e6-510a-489a-a940-9dd64ae2fb03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:40.700716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6777aa-4abe-44ef-bc4b-4234c56fb5b1 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:40.714366 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3d429-ee4a-4949-8b95-1e4d50531f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:40.715062 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3d429-ee4a-4949-8b95-1e4d50531f59 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:40.715357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3d429-ee4a-4949-8b95-1e4d50531f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:40.715612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3d429-ee4a-4949-8b95-1e4d50531f59 admin admin] PATH_INFO: `/v3/projects/be6872b7be524528b018e46b361ff259/users/0704c0407e0846aebb13cb9111e7bdc2/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:40.739933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-362621e6-510a-489a-a940-9dd64ae2fb03 None None] MFA Rules not processed for user `5f09111d61c4498eb51180fb08e19772`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:40.745601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-362621e6-510a-489a-a940-9dd64ae2fb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:40.748837 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3d429-ee4a-4949-8b95-1e4d50531f59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be6872b7be524528b018e46b361ff259, user_id=0704c0407e0846aebb13cb9111e7bdc2, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:40.753693 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3d429-ee4a-4949-8b95-1e4d50531f59 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:40.792330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 873/1767] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:40 2022] PUT /identity/v3/projects/be6872b7be524528b018e46b361ff259/users/0704c0407e0846aebb13cb9111e7bdc2/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:40.793198 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3d429-ee4a-4949-8b95-1e4d50531f59 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:40.990410 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 895/1768] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:40.992516 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362621e6-510a-489a-a940-9dd64ae2fb03 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:40.995306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3d429-ee4a-4949-8b95-1e4d50531f59 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:41.010844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8282a5-59f3-4ee3-b1b7-1344064a7121 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:41.011980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8282a5-59f3-4ee3-b1b7-1344064a7121 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:41.012261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8282a5-59f3-4ee3-b1b7-1344064a7121 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:41.012527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8282a5-59f3-4ee3-b1b7-1344064a7121 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:41.017417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362621e6-510a-489a-a940-9dd64ae2fb03 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:41.033135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:41.154095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8282a5-59f3-4ee3-b1b7-1344064a7121 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:41.155849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8282a5-59f3-4ee3-b1b7-1344064a7121 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:41.162789 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e87e88-47d9-476f-9de9-3bd0efdcb6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:41.163518 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e87e88-47d9-476f-9de9-3bd0efdcb6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:41.163872 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e87e88-47d9-476f-9de9-3bd0efdcb6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:41.164276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e87e88-47d9-476f-9de9-3bd0efdcb6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:41.164797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e87e88-47d9-476f-9de9-3bd0efdcb6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:41.167087 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e87e88-47d9-476f-9de9-3bd0efdcb6a9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:41.180016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 874/1769] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:41.181835 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed82a2c-d065-4332-b7b7-c5b194527486 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:41.182114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed82a2c-d065-4332-b7b7-c5b194527486 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:41.182390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed82a2c-d065-4332-b7b7-c5b194527486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:41.198558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 896/1770] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:41.199509 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e87e88-47d9-476f-9de9-3bd0efdcb6a9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:41.230494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e87e88-47d9-476f-9de9-3bd0efdcb6a9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:41.237256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027c274-ea78-4078-80fc-c77083655d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:41.237878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027c274-ea78-4078-80fc-c77083655d74 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:41.238164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027c274-ea78-4078-80fc-c77083655d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:41.238431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1027c274-ea78-4078-80fc-c77083655d74 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:41.239209 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027c274-ea78-4078-80fc-c77083655d74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:41.242007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1027c274-ea78-4078-80fc-c77083655d74 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:41.250789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3ed82a2c-d065-4332-b7b7-c5b194527486 None None] MFA Rules not processed for user `0704c0407e0846aebb13cb9111e7bdc2`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:41.251864 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3ed82a2c-d065-4332-b7b7-c5b194527486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:41.308093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 897/1771] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:41 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:41.309687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027c274-ea78-4078-80fc-c77083655d74 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:41.433369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 875/1772] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:41.434457 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed82a2c-d065-4332-b7b7-c5b194527486 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:41.496095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed82a2c-d065-4332-b7b7-c5b194527486 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:41.532367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:41.547223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1027c274-ea78-4078-80fc-c77083655d74 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:41.554194 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8aef05-8fac-40ae-a51a-c26455a9d9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:41.555643 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8aef05-8fac-40ae-a51a-c26455a9d9ff admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:41.556443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8aef05-8fac-40ae-a51a-c26455a9d9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:41.556914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8aef05-8fac-40ae-a51a-c26455a9d9ff admin admin] PATH_INFO: `/v3/projects/ad6335dc2fb64d03b66469e58f296822/users/7099a63ac97b4fcdbe66f94f353f0a1d/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:41.599383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8aef05-8fac-40ae-a51a-c26455a9d9ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6335dc2fb64d03b66469e58f296822, user_id=7099a63ac97b4fcdbe66f94f353f0a1d, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:41.614995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8aef05-8fac-40ae-a51a-c26455a9d9ff admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:41.648848 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 898/1773] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:41 2022] PUT /identity/v3/projects/ad6335dc2fb64d03b66469e58f296822/users/7099a63ac97b4fcdbe66f94f353f0a1d/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:41.654270 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8aef05-8fac-40ae-a51a-c26455a9d9ff admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:41.661419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8353fa-ca35-4eef-a9e1-5e5956469f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:41.662119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8353fa-ca35-4eef-a9e1-5e5956469f40 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:41.662768 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8353fa-ca35-4eef-a9e1-5e5956469f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:41.663667 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8353fa-ca35-4eef-a9e1-5e5956469f40 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:41.665694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8353fa-ca35-4eef-a9e1-5e5956469f40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:41.670496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8353fa-ca35-4eef-a9e1-5e5956469f40 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:41.748267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 876/1774] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:41 2022] POST /identity/v3/users => generated 296 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:41.749278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8353fa-ca35-4eef-a9e1-5e5956469f40 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:41.766905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8353fa-ca35-4eef-a9e1-5e5956469f40 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:41.876403 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8aef05-8fac-40ae-a51a-c26455a9d9ff admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:41.901909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ae230-6ae8-47fb-be98-350da613e684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:41.903772 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ae230-6ae8-47fb-be98-350da613e684 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:41.904569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ae230-6ae8-47fb-be98-350da613e684 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:41.904879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ae230-6ae8-47fb-be98-350da613e684 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:41.907429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ae230-6ae8-47fb-be98-350da613e684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:41.913273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ae230-6ae8-47fb-be98-350da613e684 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:41.915311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:41.965697 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cce88c-3d3d-48a3-98a0-1446319526da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:41.969992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cce88c-3d3d-48a3-98a0-1446319526da admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:41.969992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cce88c-3d3d-48a3-98a0-1446319526da admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:41.970624 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cce88c-3d3d-48a3-98a0-1446319526da admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:41.971426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cce88c-3d3d-48a3-98a0-1446319526da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:41.974175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cce88c-3d3d-48a3-98a0-1446319526da admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:42.005455 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 899/1775] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:41 2022] POST /identity/v3/projects => generated 377 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:42.006505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ae230-6ae8-47fb-be98-350da613e684 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:42.009869 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 877/1776] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:42.010932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cce88c-3d3d-48a3-98a0-1446319526da admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:42.037674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cce88c-3d3d-48a3-98a0-1446319526da admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:42.147433 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ae230-6ae8-47fb-be98-350da613e684 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:42.162667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d1cb3-2834-445f-b8c6-31706941ccff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:42.163346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d1cb3-2834-445f-b8c6-31706941ccff admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:42.163590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d1cb3-2834-445f-b8c6-31706941ccff admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:42.164536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d1cb3-2834-445f-b8c6-31706941ccff admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:42.164536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d1cb3-2834-445f-b8c6-31706941ccff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:42.173007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d1cb3-2834-445f-b8c6-31706941ccff admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:42.186253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 900/1777] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:42.187317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d1cb3-2834-445f-b8c6-31706941ccff admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:42.204416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2652bf2-eb05-4a89-8c31-9e4ccb652b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:42.205235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2652bf2-eb05-4a89-8c31-9e4ccb652b0a admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:42.206138 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2652bf2-eb05-4a89-8c31-9e4ccb652b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:42.206481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2652bf2-eb05-4a89-8c31-9e4ccb652b0a admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:42.207438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d1cb3-2834-445f-b8c6-31706941ccff admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:42.212638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2652bf2-eb05-4a89-8c31-9e4ccb652b0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:42.214850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2652bf2-eb05-4a89-8c31-9e4ccb652b0a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:42.219901 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcb40c-2b4f-4cf5-a900-48104d684747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:42.220592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcb40c-2b4f-4cf5-a900-48104d684747 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:42.220870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcb40c-2b4f-4cf5-a900-48104d684747 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:42.221143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddcb40c-2b4f-4cf5-a900-48104d684747 admin admin] PATH_INFO: `/v3/projects/ad6335dc2fb64d03b66469e58f296822/users/7099a63ac97b4fcdbe66f94f353f0a1d/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:42.229704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddcb40c-2b4f-4cf5-a900-48104d684747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6335dc2fb64d03b66469e58f296822, user_id=7099a63ac97b4fcdbe66f94f353f0a1d, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:42.232441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddcb40c-2b4f-4cf5-a900-48104d684747 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:42.260950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 901/1778] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:42 2022] PUT /identity/v3/projects/ad6335dc2fb64d03b66469e58f296822/users/7099a63ac97b4fcdbe66f94f353f0a1d/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:42.261778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcb40c-2b4f-4cf5-a900-48104d684747 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:42.271605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 878/1779] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:42 2022] POST /identity/v3/users => generated 289 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:42.273229 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2652bf2-eb05-4a89-8c31-9e4ccb652b0a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:42.284541 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2652bf2-eb05-4a89-8c31-9e4ccb652b0a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:42.356881 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e072997-8fd1-490a-bf4c-01a04d10535d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:42.357524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e072997-8fd1-490a-bf4c-01a04d10535d admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:42.357808 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e072997-8fd1-490a-bf4c-01a04d10535d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:42.358081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e072997-8fd1-490a-bf4c-01a04d10535d admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:42.358741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e072997-8fd1-490a-bf4c-01a04d10535d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:42.363400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e072997-8fd1-490a-bf4c-01a04d10535d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:42.382509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 879/1780] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:42.384187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad8d47-2336-43bb-86d3-114ebeae91ca None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:42.384480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad8d47-2336-43bb-86d3-114ebeae91ca None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:42.384761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad8d47-2336-43bb-86d3-114ebeae91ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:42.389951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:42.419121 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f3ad8d47-2336-43bb-86d3-114ebeae91ca None None] MFA Rules not processed for user `7099a63ac97b4fcdbe66f94f353f0a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:42.420581 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f3ad8d47-2336-43bb-86d3-114ebeae91ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:42.444382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddcb40c-2b4f-4cf5-a900-48104d684747 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:42.463533 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb769dbf-7ad3-4fb0-a961-1805f9cf603f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:42.464291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb769dbf-7ad3-4fb0-a961-1805f9cf603f admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:42.466469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb769dbf-7ad3-4fb0-a961-1805f9cf603f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:42.466761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb769dbf-7ad3-4fb0-a961-1805f9cf603f admin admin] PATH_INFO: `/v3/projects/7d848eb6c172430bb3c1412b782da093/users/b220e2d926324737bbb9d7391d25db62/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:42.487467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:42.497333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb769dbf-7ad3-4fb0-a961-1805f9cf603f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d848eb6c172430bb3c1412b782da093, user_id=b220e2d926324737bbb9d7391d25db62, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:42.501560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb769dbf-7ad3-4fb0-a961-1805f9cf603f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:42.557146 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 880/1781] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:42 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:42.558118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad8d47-2336-43bb-86d3-114ebeae91ca None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:42.572642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 902/1782] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:42 2022] PUT /identity/v3/projects/7d848eb6c172430bb3c1412b782da093/users/b220e2d926324737bbb9d7391d25db62/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:42.577101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb769dbf-7ad3-4fb0-a961-1805f9cf603f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:42.582845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad8d47-2336-43bb-86d3-114ebeae91ca None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:42.585452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fb769dbf-7ad3-4fb0-a961-1805f9cf603f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:42.762524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cce07-6af6-45a1-8619-14eaf11bb7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:42.763167 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cce07-6af6-45a1-8619-14eaf11bb7ba admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:42.763466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cce07-6af6-45a1-8619-14eaf11bb7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:42.763735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cce07-6af6-45a1-8619-14eaf11bb7ba admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:42.764537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cce07-6af6-45a1-8619-14eaf11bb7ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:42.766108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cce07-6af6-45a1-8619-14eaf11bb7ba admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:42.783217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb769dbf-7ad3-4fb0-a961-1805f9cf603f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:42.802922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e1cbb-c0c1-4908-8633-b1ba37a1175e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:42.804638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 881/1783] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:42.807207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cce07-6af6-45a1-8619-14eaf11bb7ba admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:42.818318 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e1cbb-c0c1-4908-8633-b1ba37a1175e service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:42.818536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e1cbb-c0c1-4908-8633-b1ba37a1175e service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:42.818896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e1cbb-c0c1-4908-8633-b1ba37a1175e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:42.833714 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cce07-6af6-45a1-8619-14eaf11bb7ba admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:42.861797 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d351056-9e27-4cd9-bdc6-5e2ecec1abe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:42.862632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d351056-9e27-4cd9-bdc6-5e2ecec1abe8 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:42.863047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d351056-9e27-4cd9-bdc6-5e2ecec1abe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:42.863539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d351056-9e27-4cd9-bdc6-5e2ecec1abe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:42.864392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d351056-9e27-4cd9-bdc6-5e2ecec1abe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:42.866434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d351056-9e27-4cd9-bdc6-5e2ecec1abe8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:42.900447 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 882/1784] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:42 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:42.901682 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d351056-9e27-4cd9-bdc6-5e2ecec1abe8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:42.926836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d351056-9e27-4cd9-bdc6-5e2ecec1abe8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:42.941000 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da8e0c-ed83-41bc-9b61-06c20534d3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:42.942032 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da8e0c-ed83-41bc-9b61-06c20534d3a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:42.944861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da8e0c-ed83-41bc-9b61-06c20534d3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:42.951980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da8e0c-ed83-41bc-9b61-06c20534d3a6 admin admin] PATH_INFO: `/v3/projects/b7c1b88a39b44284b3d3cf62d3d9849d/users/b0bd51757efa42898cf6c90a5e478ff3/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:42.996713 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da8e0c-ed83-41bc-9b61-06c20534d3a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7c1b88a39b44284b3d3cf62d3d9849d, user_id=b0bd51757efa42898cf6c90a5e478ff3, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:42.999868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da8e0c-ed83-41bc-9b61-06c20534d3a6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:43.058286 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 883/1785] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:42 2022] PUT /identity/v3/projects/b7c1b88a39b44284b3d3cf62d3d9849d/users/b0bd51757efa42898cf6c90a5e478ff3/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:43.076262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da8e0c-ed83-41bc-9b61-06c20534d3a6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:43.121460 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e1cbb-c0c1-4908-8633-b1ba37a1175e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:43.125128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e1cbb-c0c1-4908-8633-b1ba37a1175e service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:43.169737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 903/1786] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:42 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:43.174478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4105e-c50a-4652-b4cd-e8c5c83988d3 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:43.175455 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4105e-c50a-4652-b4cd-e8c5c83988d3 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:43.176076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4105e-c50a-4652-b4cd-e8c5c83988d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:43.253334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b1e4105e-c50a-4652-b4cd-e8c5c83988d3 None None] MFA Rules not processed for user `b0bd51757efa42898cf6c90a5e478ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:43.256074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b1e4105e-c50a-4652-b4cd-e8c5c83988d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:43.323409 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da8e0c-ed83-41bc-9b61-06c20534d3a6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:43.341646 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f6dcf-3224-4bc3-9150-582ddec994dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:43.343070 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f6dcf-3224-4bc3-9150-582ddec994dc admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:43.343070 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f6dcf-3224-4bc3-9150-582ddec994dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:43.343411 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f6dcf-3224-4bc3-9150-582ddec994dc admin admin] PATH_INFO: `/v3/projects/7d848eb6c172430bb3c1412b782da093/users/b220e2d926324737bbb9d7391d25db62/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:43.351902 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f6dcf-3224-4bc3-9150-582ddec994dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d848eb6c172430bb3c1412b782da093, user_id=b220e2d926324737bbb9d7391d25db62, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:43.377379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f6dcf-3224-4bc3-9150-582ddec994dc admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:43.437113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 884/1787] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:43 2022] PUT /identity/v3/projects/7d848eb6c172430bb3c1412b782da093/users/b220e2d926324737bbb9d7391d25db62/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:43.439095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f6dcf-3224-4bc3-9150-582ddec994dc admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:43.449102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ba3f6dcf-3224-4bc3-9150-582ddec994dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:43.513634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 904/1788] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:43.519371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97a9ba-9cbf-4722-8267-e988e0d39184 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:43.524955 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97a9ba-9cbf-4722-8267-e988e0d39184 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:43.525241 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c97a9ba-9cbf-4722-8267-e988e0d39184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:43.580534 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:43.590474 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:43.626378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-7c97a9ba-9cbf-4722-8267-e988e0d39184 None None] MFA Rules not processed for user `b220e2d926324737bbb9d7391d25db62`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:43.635191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f6dcf-3224-4bc3-9150-582ddec994dc admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:43.645426 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7c97a9ba-9cbf-4722-8267-e988e0d39184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:43.649867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f15112-6838-4db5-82fa-3ed8d767f289 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:43.650534 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f15112-6838-4db5-82fa-3ed8d767f289 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:43.650819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f15112-6838-4db5-82fa-3ed8d767f289 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:43.651096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f15112-6838-4db5-82fa-3ed8d767f289 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:43.813793 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 905/1789] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:43 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:43.814733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97a9ba-9cbf-4722-8267-e988e0d39184 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:43.845065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f15112-6838-4db5-82fa-3ed8d767f289 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:43.846571 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f15112-6838-4db5-82fa-3ed8d767f289 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:43.890127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 885/1790] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:43.891167 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f15112-6838-4db5-82fa-3ed8d767f289 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:43.930159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f15112-6838-4db5-82fa-3ed8d767f289 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:43.941702 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96a698-7b0f-4715-a611-37b9932b0436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:43.942328 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96a698-7b0f-4715-a611-37b9932b0436 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:43.942613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96a698-7b0f-4715-a611-37b9932b0436 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:43.942885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96a698-7b0f-4715-a611-37b9932b0436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:43.958572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c97a9ba-9cbf-4722-8267-e988e0d39184 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:43.969041 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c5145-ae23-47a9-9a85-5124cd6c0469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:43.971233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c5145-ae23-47a9-9a85-5124cd6c0469 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:43.971642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c5145-ae23-47a9-9a85-5124cd6c0469 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:43.977259 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c5145-ae23-47a9-9a85-5124cd6c0469 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:43.980297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96a698-7b0f-4715-a611-37b9932b0436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:43.980592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c5145-ae23-47a9-9a85-5124cd6c0469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:43.981842 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96a698-7b0f-4715-a611-37b9932b0436 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:43.986265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c5145-ae23-47a9-9a85-5124cd6c0469 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:44.019423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 886/1791] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:43 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:44.020756 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6892e-9c3b-481c-bb10-a01431730ef3 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:44.021041 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6892e-9c3b-481c-bb10-a01431730ef3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:44.021312 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6892e-9c3b-481c-bb10-a01431730ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:44.090169 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 906/1792] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:52:43 2022] POST /identity/v3/projects => generated 379 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:44.091309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c5145-ae23-47a9-9a85-5124cd6c0469 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:44.273442 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c5145-ae23-47a9-9a85-5124cd6c0469 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:44.284545 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-69d6892e-9c3b-481c-bb10-a01431730ef3 None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:44.288614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4e4bc-e268-4dee-85d8-42950200a1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:44.289030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-69d6892e-9c3b-481c-bb10-a01431730ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:44.290217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4e4bc-e268-4dee-85d8-42950200a1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:44.290217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4e4bc-e268-4dee-85d8-42950200a1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:44.290217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d4e4bc-e268-4dee-85d8-42950200a1e4 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:44.291012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4e4bc-e268-4dee-85d8-42950200a1e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:44.293003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d4e4bc-e268-4dee-85d8-42950200a1e4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:44.365150 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 907/1793] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:52:44 2022] POST /identity/v3/users => generated 296 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:52:44.368729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4e4bc-e268-4dee-85d8-42950200a1e4 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:44.409299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:44.502748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d4e4bc-e268-4dee-85d8-42950200a1e4 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:44.515250 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aa16a-ec37-417a-8faf-584084672203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:44.515930 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aa16a-ec37-417a-8faf-584084672203 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:44.516217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aa16a-ec37-417a-8faf-584084672203 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:44.516493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394aa16a-ec37-417a-8faf-584084672203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:44.637646 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 887/1794] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Sep 19 14:52:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 617 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:44.639462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6892e-9c3b-481c-bb10-a01431730ef3 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:44.694231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6892e-9c3b-481c-bb10-a01431730ef3 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:44.737356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a8f98-a72d-40d1-9177-9637709b1301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:44.738559 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a8f98-a72d-40d1-9177-9637709b1301 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:44.739552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a8f98-a72d-40d1-9177-9637709b1301 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:44.740952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a8f98-a72d-40d1-9177-9637709b1301 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:44.741662 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a8f98-a72d-40d1-9177-9637709b1301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:44.746808 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a8f98-a72d-40d1-9177-9637709b1301 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:44.790476 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 888/1795] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:52:44.798374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8deec1-7c9b-4dc1-b42b-237ec2e08478 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:44.799413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8deec1-7c9b-4dc1-b42b-237ec2e08478 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:44.802441 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8deec1-7c9b-4dc1-b42b-237ec2e08478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:44.817442 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394aa16a-ec37-417a-8faf-584084672203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:44.818441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394aa16a-ec37-417a-8faf-584084672203 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:44.853285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 908/1796] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:44.856002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aa16a-ec37-417a-8faf-584084672203 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:44.881159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394aa16a-ec37-417a-8faf-584084672203 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:44.890856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad98c1-5a3b-4127-9846-0f58465b449e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:44.891661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad98c1-5a3b-4127-9846-0f58465b449e admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:44.891977 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad98c1-5a3b-4127-9846-0f58465b449e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:44.892258 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad98c1-5a3b-4127-9846-0f58465b449e admin admin] PATH_INFO: `/v3/projects/e08cc2c32bf4488381d27b4212bddb76/users/6dc256d963d5434caf3d02486a1c83bd/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:44.920881 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ff8deec1-7c9b-4dc1-b42b-237ec2e08478 None None] MFA Rules not processed for user `61743a63def7451d9b739f3b17afe8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:44.929371 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ff8deec1-7c9b-4dc1-b42b-237ec2e08478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:44.958578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad98c1-5a3b-4127-9846-0f58465b449e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e08cc2c32bf4488381d27b4212bddb76, user_id=6dc256d963d5434caf3d02486a1c83bd, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:44.962076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad98c1-5a3b-4127-9846-0f58465b449e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:44.988631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 909/1797] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:44 2022] PUT /identity/v3/projects/e08cc2c32bf4488381d27b4212bddb76/users/6dc256d963d5434caf3d02486a1c83bd/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:44.992876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad98c1-5a3b-4127-9846-0f58465b449e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:45.037352 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:45.152164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 889/1798] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:45.153186 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8deec1-7c9b-4dc1-b42b-237ec2e08478 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:45.154625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ff8deec1-7c9b-4dc1-b42b-237ec2e08478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:45.157260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad98c1-5a3b-4127-9846-0f58465b449e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:45.170808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799d3f2-12c4-42d2-8c32-7082bcf91e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:45.172005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799d3f2-12c4-42d2-8c32-7082bcf91e64 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:45.172577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799d3f2-12c4-42d2-8c32-7082bcf91e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:45.173876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799d3f2-12c4-42d2-8c32-7082bcf91e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:45.174989 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799d3f2-12c4-42d2-8c32-7082bcf91e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:45.179324 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799d3f2-12c4-42d2-8c32-7082bcf91e64 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:45.195995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 910/1799] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:52:44 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:52:45.234846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799d3f2-12c4-42d2-8c32-7082bcf91e64 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:45.270379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799d3f2-12c4-42d2-8c32-7082bcf91e64 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:45.285318 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e035d-58fa-4de2-a5ab-9ef261fe65bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:45.286160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e035d-58fa-4de2-a5ab-9ef261fe65bb admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:45.286447 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e035d-58fa-4de2-a5ab-9ef261fe65bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:45.286717 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e035d-58fa-4de2-a5ab-9ef261fe65bb admin admin] PATH_INFO: `/v3/projects/e08cc2c32bf4488381d27b4212bddb76/users/6dc256d963d5434caf3d02486a1c83bd/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:45.290109 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e035d-58fa-4de2-a5ab-9ef261fe65bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e08cc2c32bf4488381d27b4212bddb76, user_id=6dc256d963d5434caf3d02486a1c83bd, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:45.293225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527e035d-58fa-4de2-a5ab-9ef261fe65bb admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:45.326531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 911/1800] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:52:45 2022] PUT /identity/v3/projects/e08cc2c32bf4488381d27b4212bddb76/users/6dc256d963d5434caf3d02486a1c83bd/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:45.336979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9a3f9-c940-4e2b-b99d-75c6b21b0f92 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:45.337451 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9a3f9-c940-4e2b-b99d-75c6b21b0f92 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:45.337867 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9a3f9-c940-4e2b-b99d-75c6b21b0f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:45.420499 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8deec1-7c9b-4dc1-b42b-237ec2e08478 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:45.422233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-09c9a3f9-c940-4e2b-b99d-75c6b21b0f92 None None] MFA Rules not processed for user `6dc256d963d5434caf3d02486a1c83bd`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:45.425307 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-09c9a3f9-c940-4e2b-b99d-75c6b21b0f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:45.454549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:45.590470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d049df-dcc3-4f7b-b133-ddb8c1bb906f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:45.591277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d049df-dcc3-4f7b-b133-ddb8c1bb906f service swift] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:45.591698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d049df-dcc3-4f7b-b133-ddb8c1bb906f service swift] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:45.592107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d049df-dcc3-4f7b-b133-ddb8c1bb906f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:45.656052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 912/1801] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:45.668803 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9a3f9-c940-4e2b-b99d-75c6b21b0f92 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:45.745100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d049df-dcc3-4f7b-b133-ddb8c1bb906f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:45.747617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d049df-dcc3-4f7b-b133-ddb8c1bb906f service swift] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:45.769893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 890/1802] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 14:52:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 14:52:45.846448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9a3f9-c940-4e2b-b99d-75c6b21b0f92 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:45.853689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a51d3d-d651-432c-88f6-353a467e8924 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:45.854336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a51d3d-d651-432c-88f6-353a467e8924 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:45.854789 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a51d3d-d651-432c-88f6-353a467e8924 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:45.855611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a51d3d-d651-432c-88f6-353a467e8924 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:45.884431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a51d3d-d651-432c-88f6-353a467e8924 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:45.889106 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a51d3d-d651-432c-88f6-353a467e8924 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:45.923888 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 913/1803] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:52:46.005970 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d049df-dcc3-4f7b-b133-ddb8c1bb906f service swift] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:46.013327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:46.023552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d049df-dcc3-4f7b-b133-ddb8c1bb906f service swift] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:46.044974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5804d-076a-442b-9482-344af3e8f4af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:46.046246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5804d-076a-442b-9482-344af3e8f4af service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:46.046548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5804d-076a-442b-9482-344af3e8f4af service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:46.046883 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5804d-076a-442b-9482-344af3e8f4af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:46.197480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5804d-076a-442b-9482-344af3e8f4af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:46.199358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5804d-076a-442b-9482-344af3e8f4af service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:46.239173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 891/1804] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:50.162748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76227ff3-08bb-4431-a997-ee11736e3ad8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:50.163723 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76227ff3-08bb-4431-a997-ee11736e3ad8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:50.164178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76227ff3-08bb-4431-a997-ee11736e3ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:50.226544 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-76227ff3-08bb-4431-a997-ee11736e3ad8 None None] MFA Rules not processed for user `2fbef7b24f134f5eb1193aed1b7ee69c`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:50.227867 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-76227ff3-08bb-4431-a997-ee11736e3ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:50.433760 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 914/1805] 10.222.0.71 () {58 vars in 1015 bytes} [Mon Sep 19 14:52:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:50.476102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5804d-076a-442b-9482-344af3e8f4af service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:50.496311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5804d-076a-442b-9482-344af3e8f4af service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:50.504256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76227ff3-08bb-4431-a997-ee11736e3ad8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:50.518809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d9c30-98a7-4e42-a9e5-4a9227bfa010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:50.519855 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d9c30-98a7-4e42-a9e5-4a9227bfa010 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:50.521777 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76227ff3-08bb-4431-a997-ee11736e3ad8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:50.522421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d9c30-98a7-4e42-a9e5-4a9227bfa010 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:50.522891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d9c30-98a7-4e42-a9e5-4a9227bfa010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:50.529760 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65eb0c-651e-4435-af4d-66759d049c33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:50.530420 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65eb0c-651e-4435-af4d-66759d049c33 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:50.532345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65eb0c-651e-4435-af4d-66759d049c33 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:50.533348 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65eb0c-651e-4435-af4d-66759d049c33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:50.546304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d9c30-98a7-4e42-a9e5-4a9227bfa010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:50.547026 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65eb0c-651e-4435-af4d-66759d049c33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:50.549305 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65eb0c-651e-4435-af4d-66759d049c33 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:50.557721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d9c30-98a7-4e42-a9e5-4a9227bfa010 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:50.572887 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 915/1806] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:50.589551 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 892/1807] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:51.928141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647fc1f-5908-45de-9f46-3dc5449c7515 None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:51.928928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647fc1f-5908-45de-9f46-3dc5449c7515 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:51.929245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7647fc1f-5908-45de-9f46-3dc5449c7515 None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:51.930874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 916/1808] 10.222.0.60 () {54 vars in 921 bytes} [Mon Sep 19 14:52:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:52:51.953443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46491782-2ca3-4417-bb94-71fd4eb37660 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:51.953992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46491782-2ca3-4417-bb94-71fd4eb37660 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:51.954214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46491782-2ca3-4417-bb94-71fd4eb37660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:51.976201 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-46491782-2ca3-4417-bb94-71fd4eb37660 None None] MFA Rules not processed for user `59b5d2fd362b417c9b9be75f88a667ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:51.977970 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-46491782-2ca3-4417-bb94-71fd4eb37660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:52.050508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 893/1809] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Sep 19 14:52:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:52.250251 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647fc1f-5908-45de-9f46-3dc5449c7515 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:52.252099 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7647fc1f-5908-45de-9f46-3dc5449c7515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:52.271298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7647fc1f-5908-45de-9f46-3dc5449c7515 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:52.280558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d128e69-e17a-4bbb-8cd4-671fe55df4d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:52.281345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d128e69-e17a-4bbb-8cd4-671fe55df4d5 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:52.281636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d128e69-e17a-4bbb-8cd4-671fe55df4d5 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:52.281917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d128e69-e17a-4bbb-8cd4-671fe55df4d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:52.292158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d128e69-e17a-4bbb-8cd4-671fe55df4d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:52.293751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d128e69-e17a-4bbb-8cd4-671fe55df4d5 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:52.304785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 917/1810] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:54.089171 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46491782-2ca3-4417-bb94-71fd4eb37660 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:54.211882 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46491782-2ca3-4417-bb94-71fd4eb37660 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:54.218724 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fe8e7-660b-49e7-8536-2a7feba95d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:54.219730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fe8e7-660b-49e7-8536-2a7feba95d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:54.222115 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fe8e7-660b-49e7-8536-2a7feba95d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:54.222623 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fe8e7-660b-49e7-8536-2a7feba95d54 admin admin] PATH_INFO: `/v3/users/0704c0407e0846aebb13cb9111e7bdc2` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:54.230356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fe8e7-660b-49e7-8536-2a7feba95d54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0704c0407e0846aebb13cb9111e7bdc2)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:54.233238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fe8e7-660b-49e7-8536-2a7feba95d54 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:54.301644 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:54.354143 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c36fe8e7-660b-49e7-8536-2a7feba95d54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0704c0407e0846aebb13cb9111e7bdc2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:54.379257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c36fe8e7-660b-49e7-8536-2a7feba95d54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0704c0407e0846aebb13cb9111e7bdc2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:54.423089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c36fe8e7-660b-49e7-8536-2a7feba95d54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0704c0407e0846aebb13cb9111e7bdc2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:54.456331 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c36fe8e7-660b-49e7-8536-2a7feba95d54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0704c0407e0846aebb13cb9111e7bdc2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:54.457274 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c36fe8e7-660b-49e7-8536-2a7feba95d54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0704c0407e0846aebb13cb9111e7bdc2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:54.481690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 894/1811] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:52:54 2022] DELETE /identity/v3/users/0704c0407e0846aebb13cb9111e7bdc2 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:54.486509 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25e467-5045-448e-a1b3-a44802312b5f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:54.487062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25e467-5045-448e-a1b3-a44802312b5f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:54.487254 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e25e467-5045-448e-a1b3-a44802312b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:54.521234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8e25e467-5045-448e-a1b3-a44802312b5f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:54.522490 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8e25e467-5045-448e-a1b3-a44802312b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:54.575282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:54.617695 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 918/1812] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:54.629144 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fe8e7-660b-49e7-8536-2a7feba95d54 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:54.630717 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c36fe8e7-660b-49e7-8536-2a7feba95d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:54.730083 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fe8e7-660b-49e7-8536-2a7feba95d54 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:54.735852 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114f853-520e-4ad3-a425-a47f6ae9da69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:54.736899 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114f853-520e-4ad3-a425-a47f6ae9da69 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:54.737245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114f853-520e-4ad3-a425-a47f6ae9da69 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:54.738027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114f853-520e-4ad3-a425-a47f6ae9da69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:54.753701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114f853-520e-4ad3-a425-a47f6ae9da69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:54.757673 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114f853-520e-4ad3-a425-a47f6ae9da69 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:54.797688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 895/1813] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:55.241857 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25e467-5045-448e-a1b3-a44802312b5f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:55.243699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8e25e467-5045-448e-a1b3-a44802312b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:55.258790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e25e467-5045-448e-a1b3-a44802312b5f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:55.267764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fc04b-0b89-411e-bff6-5065001afeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:55.272267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fc04b-0b89-411e-bff6-5065001afeba admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:55.272901 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fc04b-0b89-411e-bff6-5065001afeba admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:55.273242 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fc04b-0b89-411e-bff6-5065001afeba admin admin] PATH_INFO: `/v3/projects/be6872b7be524528b018e46b361ff259` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:55.275091 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308fc04b-0b89-411e-bff6-5065001afeba admin admin] RBAC: Authorizing `identity:delete_project(project_id=be6872b7be524528b018e46b361ff259)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:55.277056 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308fc04b-0b89-411e-bff6-5065001afeba admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:55.338445 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114f853-520e-4ad3-a425-a47f6ae9da69 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:55.340030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b114f853-520e-4ad3-a425-a47f6ae9da69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:55.367906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114f853-520e-4ad3-a425-a47f6ae9da69 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:55.415575 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-308fc04b-0b89-411e-bff6-5065001afeba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be6872b7be524528b018e46b361ff259', 'request_id': 'req-308fc04b-0b89-411e-bff6-5065001afeba'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:55.441458 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-308fc04b-0b89-411e-bff6-5065001afeba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be6872b7be524528b018e46b361ff259', 'request_id': 'req-308fc04b-0b89-411e-bff6-5065001afeba'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:55.468805 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-308fc04b-0b89-411e-bff6-5065001afeba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be6872b7be524528b018e46b361ff259', 'request_id': 'req-308fc04b-0b89-411e-bff6-5065001afeba'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:55.484784 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-308fc04b-0b89-411e-bff6-5065001afeba admin admin] The token cache is being invalidate because project be6872b7be524528b018e46b361ff259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:52:55.484784 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-308fc04b-0b89-411e-bff6-5065001afeba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:55.486585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-308fc04b-0b89-411e-bff6-5065001afeba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:55.491577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 919/1814] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:52:55 2022] DELETE /identity/v3/projects/be6872b7be524528b018e46b361ff259 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:55.493752 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fc04b-0b89-411e-bff6-5065001afeba admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:55.495810 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-308fc04b-0b89-411e-bff6-5065001afeba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:55.507229 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e35c3-bf98-4d69-a81f-6bf304492b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:55.508355 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e35c3-bf98-4d69-a81f-6bf304492b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:55.508804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e35c3-bf98-4d69-a81f-6bf304492b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:55.509239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e35c3-bf98-4d69-a81f-6bf304492b1c admin admin] PATH_INFO: `/v3/users/0517c431d8f6424289aab070af2acee6` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:55.510657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e35c3-bf98-4d69-a81f-6bf304492b1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0517c431d8f6424289aab070af2acee6)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:55.513279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e35c3-bf98-4d69-a81f-6bf304492b1c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:55.557246 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fc04b-0b89-411e-bff6-5065001afeba admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:55.571650 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aec108-b775-4bd7-bf86-241327e6a8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:55.573859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aec108-b775-4bd7-bf86-241327e6a8b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:55.574762 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aec108-b775-4bd7-bf86-241327e6a8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:55.575541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aec108-b775-4bd7-bf86-241327e6a8b4 admin admin] PATH_INFO: `/v3/users/b0bd51757efa42898cf6c90a5e478ff3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:55.579061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aec108-b775-4bd7-bf86-241327e6a8b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0bd51757efa42898cf6c90a5e478ff3)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:55.584748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aec108-b775-4bd7-bf86-241327e6a8b4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:55.625465 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:55.629222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-765e35c3-bf98-4d69-a81f-6bf304492b1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0517c431d8f6424289aab070af2acee6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:55.667767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-765e35c3-bf98-4d69-a81f-6bf304492b1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0517c431d8f6424289aab070af2acee6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:55.717906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-33aec108-b775-4bd7-bf86-241327e6a8b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0bd51757efa42898cf6c90a5e478ff3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:55.752621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-765e35c3-bf98-4d69-a81f-6bf304492b1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0517c431d8f6424289aab070af2acee6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:55.755425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-33aec108-b775-4bd7-bf86-241327e6a8b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0bd51757efa42898cf6c90a5e478ff3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:55.776603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-765e35c3-bf98-4d69-a81f-6bf304492b1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0517c431d8f6424289aab070af2acee6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:55.780411 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-765e35c3-bf98-4d69-a81f-6bf304492b1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0517c431d8f6424289aab070af2acee6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:55.789348 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-33aec108-b775-4bd7-bf86-241327e6a8b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0bd51757efa42898cf6c90a5e478ff3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:55.807998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 896/1815] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:52:55 2022] DELETE /identity/v3/users/0517c431d8f6424289aab070af2acee6 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:55.813018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da048f58-01fa-4695-8331-22a27678937c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:55.814604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da048f58-01fa-4695-8331-22a27678937c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:55.815193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da048f58-01fa-4695-8331-22a27678937c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:55.824796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-33aec108-b775-4bd7-bf86-241327e6a8b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0bd51757efa42898cf6c90a5e478ff3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:55.826010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-33aec108-b775-4bd7-bf86-241327e6a8b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0bd51757efa42898cf6c90a5e478ff3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:55.873623 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 920/1816] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:52:55 2022] DELETE /identity/v3/users/b0bd51757efa42898cf6c90a5e478ff3 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:55.892552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-da048f58-01fa-4695-8331-22a27678937c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:55.897574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-da048f58-01fa-4695-8331-22a27678937c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:56.071420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 897/1817] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:56.197500 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aec108-b775-4bd7-bf86-241327e6a8b4 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:56.199022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-33aec108-b775-4bd7-bf86-241327e6a8b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:56.347901 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:56.455154 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aec108-b775-4bd7-bf86-241327e6a8b4 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:56.493488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52379c1b-6ee7-4510-b7c7-320d6169d2f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:56.497045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52379c1b-6ee7-4510-b7c7-320d6169d2f1 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:56.497608 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52379c1b-6ee7-4510-b7c7-320d6169d2f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:56.498594 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52379c1b-6ee7-4510-b7c7-320d6169d2f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:56.523316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52379c1b-6ee7-4510-b7c7-320d6169d2f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:56.525560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52379c1b-6ee7-4510-b7c7-320d6169d2f1 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:56.555991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 921/1818] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:56.950729 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da048f58-01fa-4695-8331-22a27678937c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:56.952989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-da048f58-01fa-4695-8331-22a27678937c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:57.001828 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da048f58-01fa-4695-8331-22a27678937c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:57.022123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe38b4-3217-4631-ba62-b261209e3996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:57.022796 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe38b4-3217-4631-ba62-b261209e3996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:57.023084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe38b4-3217-4631-ba62-b261209e3996 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:57.023442 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe38b4-3217-4631-ba62-b261209e3996 admin admin] PATH_INFO: `/v3/projects/b7c1b88a39b44284b3d3cf62d3d9849d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:57.040956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe38b4-3217-4631-ba62-b261209e3996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7c1b88a39b44284b3d3cf62d3d9849d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:57.043118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efe38b4-3217-4631-ba62-b261209e3996 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:57.184620 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8efe38b4-3217-4631-ba62-b261209e3996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7c1b88a39b44284b3d3cf62d3d9849d', 'request_id': 'req-8efe38b4-3217-4631-ba62-b261209e3996'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:57.210387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52379c1b-6ee7-4510-b7c7-320d6169d2f1 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:57.212131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-52379c1b-6ee7-4510-b7c7-320d6169d2f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:57.223605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8efe38b4-3217-4631-ba62-b261209e3996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7c1b88a39b44284b3d3cf62d3d9849d', 'request_id': 'req-8efe38b4-3217-4631-ba62-b261209e3996'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:57.247126 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8efe38b4-3217-4631-ba62-b261209e3996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7c1b88a39b44284b3d3cf62d3d9849d', 'request_id': 'req-8efe38b4-3217-4631-ba62-b261209e3996'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:57.258881 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52379c1b-6ee7-4510-b7c7-320d6169d2f1 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:57.265038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8efe38b4-3217-4631-ba62-b261209e3996 admin admin] The token cache is being invalidate because project b7c1b88a39b44284b3d3cf62d3d9849d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:52:57.265391 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8efe38b4-3217-4631-ba62-b261209e3996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:57.266007 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8efe38b4-3217-4631-ba62-b261209e3996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:57.267578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 898/1819] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:52:56 2022] DELETE /identity/v3/projects/b7c1b88a39b44284b3d3cf62d3d9849d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:57.271600 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe38b4-3217-4631-ba62-b261209e3996 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:57.273041 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8efe38b4-3217-4631-ba62-b261209e3996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:57.369023 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712909bf-563b-4a57-940a-86c38897b239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:57.369876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712909bf-563b-4a57-940a-86c38897b239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:57.370272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712909bf-563b-4a57-940a-86c38897b239 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:57.370622 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712909bf-563b-4a57-940a-86c38897b239 admin admin] PATH_INFO: `/v3/projects/e185bf96475e47d3a0ec27f8185ddbe9` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:57.372185 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712909bf-563b-4a57-940a-86c38897b239 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e185bf96475e47d3a0ec27f8185ddbe9)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:57.375053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712909bf-563b-4a57-940a-86c38897b239 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:57.394212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:57.420078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe38b4-3217-4631-ba62-b261209e3996 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:57.437777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e52d9-f89c-452d-a399-810f36b72a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:57.438531 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e52d9-f89c-452d-a399-810f36b72a56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:57.439489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e52d9-f89c-452d-a399-810f36b72a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:57.439918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e52d9-f89c-452d-a399-810f36b72a56 admin admin] PATH_INFO: `/v3/users/6dc256d963d5434caf3d02486a1c83bd` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:57.442380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e52d9-f89c-452d-a399-810f36b72a56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dc256d963d5434caf3d02486a1c83bd)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:57.450222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e52d9-f89c-452d-a399-810f36b72a56 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:57.464508 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-712909bf-563b-4a57-940a-86c38897b239 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e185bf96475e47d3a0ec27f8185ddbe9', 'request_id': 'req-712909bf-563b-4a57-940a-86c38897b239'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:57.475193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-712909bf-563b-4a57-940a-86c38897b239 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e185bf96475e47d3a0ec27f8185ddbe9', 'request_id': 'req-712909bf-563b-4a57-940a-86c38897b239'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:57.490444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-712909bf-563b-4a57-940a-86c38897b239 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e185bf96475e47d3a0ec27f8185ddbe9', 'request_id': 'req-712909bf-563b-4a57-940a-86c38897b239'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:57.527198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-712909bf-563b-4a57-940a-86c38897b239 admin admin] The token cache is being invalidate because project e185bf96475e47d3a0ec27f8185ddbe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:52:57.528282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-712909bf-563b-4a57-940a-86c38897b239 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:57.529975 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-712909bf-563b-4a57-940a-86c38897b239 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:57.534931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 922/1820] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:52:57 2022] DELETE /identity/v3/projects/e185bf96475e47d3a0ec27f8185ddbe9 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:57.539299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712909bf-563b-4a57-940a-86c38897b239 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:57.542191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-712909bf-563b-4a57-940a-86c38897b239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:57.583779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-613e52d9-f89c-452d-a399-810f36b72a56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dc256d963d5434caf3d02486a1c83bd'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:57.604064 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-613e52d9-f89c-452d-a399-810f36b72a56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dc256d963d5434caf3d02486a1c83bd'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:57.634041 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-613e52d9-f89c-452d-a399-810f36b72a56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dc256d963d5434caf3d02486a1c83bd'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:57.663564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-613e52d9-f89c-452d-a399-810f36b72a56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dc256d963d5434caf3d02486a1c83bd'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:57.664718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-613e52d9-f89c-452d-a399-810f36b72a56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dc256d963d5434caf3d02486a1c83bd'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:57.685568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 899/1821] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:52:57 2022] DELETE /identity/v3/users/6dc256d963d5434caf3d02486a1c83bd => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:52:57.691167 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e52d9-f89c-452d-a399-810f36b72a56 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:57.696985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-613e52d9-f89c-452d-a399-810f36b72a56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:57.720734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712909bf-563b-4a57-940a-86c38897b239 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:57.738987 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:57.897190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90bfd0-c632-4915-b307-98460e88b9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:57.898280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90bfd0-c632-4915-b307-98460e88b9fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:57.901127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90bfd0-c632-4915-b307-98460e88b9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:57.902304 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90bfd0-c632-4915-b307-98460e88b9fa admin admin] PATH_INFO: `/v3/users/6a661ca101dd415799678eefd27aceac` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:57.905013 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90bfd0-c632-4915-b307-98460e88b9fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a661ca101dd415799678eefd27aceac)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:57.910603 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90bfd0-c632-4915-b307-98460e88b9fa admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:57.912503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e52d9-f89c-452d-a399-810f36b72a56 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:57.934849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39506c-c6c5-40b2-be51-007b3f301772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:57.939410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39506c-c6c5-40b2-be51-007b3f301772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:57.940149 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39506c-c6c5-40b2-be51-007b3f301772 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:57.941000 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39506c-c6c5-40b2-be51-007b3f301772 admin admin] PATH_INFO: `/v3/users/5f09111d61c4498eb51180fb08e19772` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:57.947041 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39506c-c6c5-40b2-be51-007b3f301772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f09111d61c4498eb51180fb08e19772)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:57.950962 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39506c-c6c5-40b2-be51-007b3f301772 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:58.053981 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7d90bfd0-c632-4915-b307-98460e88b9fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a661ca101dd415799678eefd27aceac'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:58.057404 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dd39506c-c6c5-40b2-be51-007b3f301772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f09111d61c4498eb51180fb08e19772'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:58.071933 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dd39506c-c6c5-40b2-be51-007b3f301772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f09111d61c4498eb51180fb08e19772'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:58.080175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7d90bfd0-c632-4915-b307-98460e88b9fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a661ca101dd415799678eefd27aceac'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:58.100200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:58.110043 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dd39506c-c6c5-40b2-be51-007b3f301772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f09111d61c4498eb51180fb08e19772'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:58.121738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dd39506c-c6c5-40b2-be51-007b3f301772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f09111d61c4498eb51180fb08e19772'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:58.122532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dd39506c-c6c5-40b2-be51-007b3f301772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f09111d61c4498eb51180fb08e19772'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:58.130510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7d90bfd0-c632-4915-b307-98460e88b9fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a661ca101dd415799678eefd27aceac'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:58.136302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 900/1822] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:52:57 2022] DELETE /identity/v3/users/5f09111d61c4498eb51180fb08e19772 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:58.137860 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ad4c9-b415-41ac-8869-f9646c36c513 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:58.138146 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ad4c9-b415-41ac-8869-f9646c36c513 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:58.138499 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ad4c9-b415-41ac-8869-f9646c36c513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:58.170751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7d90bfd0-c632-4915-b307-98460e88b9fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a661ca101dd415799678eefd27aceac'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:58.179969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7d90bfd0-c632-4915-b307-98460e88b9fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a661ca101dd415799678eefd27aceac'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:58.195119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 923/1823] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:52:57 2022] DELETE /identity/v3/users/6a661ca101dd415799678eefd27aceac => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:58.202376 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e16ad4c9-b415-41ac-8869-f9646c36c513 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:58.205362 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e16ad4c9-b415-41ac-8869-f9646c36c513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:58.462994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 901/1824] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:58.557240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90bfd0-c632-4915-b307-98460e88b9fa admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:58.564391 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7d90bfd0-c632-4915-b307-98460e88b9fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:58.709419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90bfd0-c632-4915-b307-98460e88b9fa admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:58.719438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758f4c3-87a2-4949-94f6-f05c0045ec9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:58.721474 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758f4c3-87a2-4949-94f6-f05c0045ec9e service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:58.721868 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758f4c3-87a2-4949-94f6-f05c0045ec9e service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:58.722248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758f4c3-87a2-4949-94f6-f05c0045ec9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:58.762754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758f4c3-87a2-4949-94f6-f05c0045ec9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:58.764893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758f4c3-87a2-4949-94f6-f05c0045ec9e service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:58.778149 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 924/1825] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:52:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:52:59.491751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ad4c9-b415-41ac-8869-f9646c36c513 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:59.494525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e16ad4c9-b415-41ac-8869-f9646c36c513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:59.520085 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ad4c9-b415-41ac-8869-f9646c36c513 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:59.529321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceffedf-162e-4c3d-ad47-53540aecf2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:59.531026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceffedf-162e-4c3d-ad47-53540aecf2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:59.531639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceffedf-162e-4c3d-ad47-53540aecf2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:59.533779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceffedf-162e-4c3d-ad47-53540aecf2f0 admin admin] PATH_INFO: `/v3/projects/e08cc2c32bf4488381d27b4212bddb76` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:59.536587 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceffedf-162e-4c3d-ad47-53540aecf2f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e08cc2c32bf4488381d27b4212bddb76)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:59.539868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceffedf-162e-4c3d-ad47-53540aecf2f0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:59.675804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fceffedf-162e-4c3d-ad47-53540aecf2f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e08cc2c32bf4488381d27b4212bddb76', 'request_id': 'req-fceffedf-162e-4c3d-ad47-53540aecf2f0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:59.692446 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fceffedf-162e-4c3d-ad47-53540aecf2f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e08cc2c32bf4488381d27b4212bddb76', 'request_id': 'req-fceffedf-162e-4c3d-ad47-53540aecf2f0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:59.704329 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fceffedf-162e-4c3d-ad47-53540aecf2f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e08cc2c32bf4488381d27b4212bddb76', 'request_id': 'req-fceffedf-162e-4c3d-ad47-53540aecf2f0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:59.722823 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fceffedf-162e-4c3d-ad47-53540aecf2f0 admin admin] The token cache is being invalidate because project e08cc2c32bf4488381d27b4212bddb76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:52:59.723156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fceffedf-162e-4c3d-ad47-53540aecf2f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:59.723808 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fceffedf-162e-4c3d-ad47-53540aecf2f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:52:59.732169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 902/1826] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:52:59 2022] DELETE /identity/v3/projects/e08cc2c32bf4488381d27b4212bddb76 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:52:59.733178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861f884-50d5-40d1-bf18-0f11a7ac5265 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:59.733669 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861f884-50d5-40d1-bf18-0f11a7ac5265 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:59.733761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861f884-50d5-40d1-bf18-0f11a7ac5265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:59.766817 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e861f884-50d5-40d1-bf18-0f11a7ac5265 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:52:59.769214 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e861f884-50d5-40d1-bf18-0f11a7ac5265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:59.791897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceffedf-162e-4c3d-ad47-53540aecf2f0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:52:59.799018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-fceffedf-162e-4c3d-ad47-53540aecf2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:52:59.811258 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:59.916796 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceffedf-162e-4c3d-ad47-53540aecf2f0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:52:59.931862 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4b566-14c6-47df-9be5-d284b2efabdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:52:59.933114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4b566-14c6-47df-9be5-d284b2efabdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:59.933114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4b566-14c6-47df-9be5-d284b2efabdf admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:59.933114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4b566-14c6-47df-9be5-d284b2efabdf admin admin] PATH_INFO: `/v3/projects/a2d404cba14e4bcf8fbcf1d6d622d22e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:52:59.934366 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4b566-14c6-47df-9be5-d284b2efabdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2d404cba14e4bcf8fbcf1d6d622d22e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:52:59.936601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4b566-14c6-47df-9be5-d284b2efabdf admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:52:59.958395 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 925/1827] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:52:59.960114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:52:59.968299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3e7e7-4a2f-4045-9577-1cf5e879d3fb None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:52:59.970153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3e7e7-4a2f-4045-9577-1cf5e879d3fb None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:52:59.970841 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3e7e7-4a2f-4045-9577-1cf5e879d3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:00.008579 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f5f3e7e7-4a2f-4045-9577-1cf5e879d3fb None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:00.014964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f5f3e7e7-4a2f-4045-9577-1cf5e879d3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:00.033951 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a0e4b566-14c6-47df-9be5-d284b2efabdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2d404cba14e4bcf8fbcf1d6d622d22e', 'request_id': 'req-a0e4b566-14c6-47df-9be5-d284b2efabdf'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:00.050012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a0e4b566-14c6-47df-9be5-d284b2efabdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2d404cba14e4bcf8fbcf1d6d622d22e', 'request_id': 'req-a0e4b566-14c6-47df-9be5-d284b2efabdf'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:00.075695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a0e4b566-14c6-47df-9be5-d284b2efabdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2d404cba14e4bcf8fbcf1d6d622d22e', 'request_id': 'req-a0e4b566-14c6-47df-9be5-d284b2efabdf'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:00.130279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a0e4b566-14c6-47df-9be5-d284b2efabdf admin admin] The token cache is being invalidate because project a2d404cba14e4bcf8fbcf1d6d622d22e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:00.130279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a0e4b566-14c6-47df-9be5-d284b2efabdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:00.132102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a0e4b566-14c6-47df-9be5-d284b2efabdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:00.139109 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 903/1828] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:52:59 2022] DELETE /identity/v3/projects/a2d404cba14e4bcf8fbcf1d6d622d22e => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:00.140093 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4b566-14c6-47df-9be5-d284b2efabdf admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:00.141707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a0e4b566-14c6-47df-9be5-d284b2efabdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:00.184192 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 926/1829] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:00.185456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3e7e7-4a2f-4045-9577-1cf5e879d3fb None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:00.199508 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3e7e7-4a2f-4045-9577-1cf5e879d3fb None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:00.205537 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12fbbe-56e3-4557-b965-d7db5ee988b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:00.206255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12fbbe-56e3-4557-b965-d7db5ee988b8 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:00.207236 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12fbbe-56e3-4557-b965-d7db5ee988b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:00.207942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12fbbe-56e3-4557-b965-d7db5ee988b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:00.208064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12fbbe-56e3-4557-b965-d7db5ee988b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:00.208372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12fbbe-56e3-4557-b965-d7db5ee988b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:00.210273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c12fbbe-56e3-4557-b965-d7db5ee988b8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:00.231045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 927/1830] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:53:00.235865 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12fbbe-56e3-4557-b965-d7db5ee988b8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:00.254398 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12fbbe-56e3-4557-b965-d7db5ee988b8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:00.264387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4a0af-b393-46f2-a0eb-879e0cdd2dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:00.265928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4a0af-b393-46f2-a0eb-879e0cdd2dfc admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:00.266378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4a0af-b393-46f2-a0eb-879e0cdd2dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:00.266799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec4a0af-b393-46f2-a0eb-879e0cdd2dfc admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:00.267636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4a0af-b393-46f2-a0eb-879e0cdd2dfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:00.270022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec4a0af-b393-46f2-a0eb-879e0cdd2dfc admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:00.282222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4b566-14c6-47df-9be5-d284b2efabdf admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:00.287724 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b2d2e-2564-430f-9ba0-58d8242c4386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:00.289216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b2d2e-2564-430f-9ba0-58d8242c4386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:00.289692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b2d2e-2564-430f-9ba0-58d8242c4386 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:00.290146 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b2d2e-2564-430f-9ba0-58d8242c4386 admin admin] PATH_INFO: `/v3/users/b220e2d926324737bbb9d7391d25db62` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:00.291887 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b2d2e-2564-430f-9ba0-58d8242c4386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b220e2d926324737bbb9d7391d25db62)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:00.294752 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b2d2e-2564-430f-9ba0-58d8242c4386 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:00.328868 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 928/1831] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:00 2022] POST /identity/v3/projects => generated 371 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:00.330174 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4a0af-b393-46f2-a0eb-879e0cdd2dfc admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:00.376553 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5c8b2d2e-2564-430f-9ba0-58d8242c4386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b220e2d926324737bbb9d7391d25db62'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:00.386944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5c8b2d2e-2564-430f-9ba0-58d8242c4386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b220e2d926324737bbb9d7391d25db62'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:00.409583 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5c8b2d2e-2564-430f-9ba0-58d8242c4386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b220e2d926324737bbb9d7391d25db62'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:00.421841 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5c8b2d2e-2564-430f-9ba0-58d8242c4386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b220e2d926324737bbb9d7391d25db62'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:00.422690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5c8b2d2e-2564-430f-9ba0-58d8242c4386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b220e2d926324737bbb9d7391d25db62'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:00.434314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 904/1832] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:00 2022] DELETE /identity/v3/users/b220e2d926324737bbb9d7391d25db62 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:00.434314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b2d2e-2564-430f-9ba0-58d8242c4386 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:00.449713 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5c8b2d2e-2564-430f-9ba0-58d8242c4386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:00.478426 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec4a0af-b393-46f2-a0eb-879e0cdd2dfc admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:00.494992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:00.558108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:00.585528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c971f9-743e-4c1e-9ba2-9bd2d7e93db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:00.586968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c971f9-743e-4c1e-9ba2-9bd2d7e93db0 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:00.587894 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c971f9-743e-4c1e-9ba2-9bd2d7e93db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:00.588281 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c971f9-743e-4c1e-9ba2-9bd2d7e93db0 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:00.589084 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c971f9-743e-4c1e-9ba2-9bd2d7e93db0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:00.591356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c971f9-743e-4c1e-9ba2-9bd2d7e93db0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:00.623560 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b2d2e-2564-430f-9ba0-58d8242c4386 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:00.626555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 929/1833] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:00 2022] POST /identity/v3/users => generated 286 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:00.631291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c971f9-743e-4c1e-9ba2-9bd2d7e93db0 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:00.633826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-45c971f9-743e-4c1e-9ba2-9bd2d7e93db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:00.640596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099daf5f-5fc5-4162-8ffb-8d56ab618057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:00.641276 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099daf5f-5fc5-4162-8ffb-8d56ab618057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:00.641557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099daf5f-5fc5-4162-8ffb-8d56ab618057 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:00.641831 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099daf5f-5fc5-4162-8ffb-8d56ab618057 admin admin] PATH_INFO: `/v3/projects/a79d80ad80824c2d8ecb9087eb03bd66` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:00.643145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099daf5f-5fc5-4162-8ffb-8d56ab618057 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a79d80ad80824c2d8ecb9087eb03bd66)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:00.645121 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099daf5f-5fc5-4162-8ffb-8d56ab618057 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:00.648697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c971f9-743e-4c1e-9ba2-9bd2d7e93db0 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:00.655387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d9c66-020d-4485-a1b9-0ae9acaf5516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:00.656820 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d9c66-020d-4485-a1b9-0ae9acaf5516 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:00.657251 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d9c66-020d-4485-a1b9-0ae9acaf5516 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:00.657602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d9c66-020d-4485-a1b9-0ae9acaf5516 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:00.658365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d9c66-020d-4485-a1b9-0ae9acaf5516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:00.660220 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d9c66-020d-4485-a1b9-0ae9acaf5516 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:00.677385 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 930/1834] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:00.678731 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d9c66-020d-4485-a1b9-0ae9acaf5516 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:00.704819 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d9c66-020d-4485-a1b9-0ae9acaf5516 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:00.715909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571f592-0387-4eda-88ab-671ead93596b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:00.717072 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571f592-0387-4eda-88ab-671ead93596b admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:00.718733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571f592-0387-4eda-88ab-671ead93596b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:00.719148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571f592-0387-4eda-88ab-671ead93596b admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:00.722198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571f592-0387-4eda-88ab-671ead93596b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:00.732002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571f592-0387-4eda-88ab-671ead93596b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:00.749467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-099daf5f-5fc5-4162-8ffb-8d56ab618057 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a79d80ad80824c2d8ecb9087eb03bd66', 'request_id': 'req-099daf5f-5fc5-4162-8ffb-8d56ab618057'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:00.757568 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 931/1835] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:00.759281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-099daf5f-5fc5-4162-8ffb-8d56ab618057 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a79d80ad80824c2d8ecb9087eb03bd66', 'request_id': 'req-099daf5f-5fc5-4162-8ffb-8d56ab618057'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:00.763576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571f592-0387-4eda-88ab-671ead93596b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:00.772032 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-099daf5f-5fc5-4162-8ffb-8d56ab618057 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a79d80ad80824c2d8ecb9087eb03bd66', 'request_id': 'req-099daf5f-5fc5-4162-8ffb-8d56ab618057'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:00.778266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571f592-0387-4eda-88ab-671ead93596b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:00.797920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-099daf5f-5fc5-4162-8ffb-8d56ab618057 admin admin] The token cache is being invalidate because project a79d80ad80824c2d8ecb9087eb03bd66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:00.810857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-099daf5f-5fc5-4162-8ffb-8d56ab618057 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:00.813751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-099daf5f-5fc5-4162-8ffb-8d56ab618057 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:00.823690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 905/1836] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:00 2022] DELETE /identity/v3/projects/a79d80ad80824c2d8ecb9087eb03bd66 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:00.830057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc89a94-f461-4280-9675-10b0f480b271 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:00.830358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc89a94-f461-4280-9675-10b0f480b271 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:00.830623 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc89a94-f461-4280-9675-10b0f480b271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:00.866703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-cdc89a94-f461-4280-9675-10b0f480b271 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:00.871436 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cdc89a94-f461-4280-9675-10b0f480b271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:00.884603 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9776544-ac6d-48bc-86f5-8ef78124477f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:00.885878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9776544-ac6d-48bc-86f5-8ef78124477f admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:00.886377 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9776544-ac6d-48bc-86f5-8ef78124477f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:00.886917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9776544-ac6d-48bc-86f5-8ef78124477f admin admin] PATH_INFO: `/v3/projects/c50ff4383b5c43788acb95c7c0a3d56f/users/5839cda8aa894d0093782221b42dbb50/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:00.907653 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9776544-ac6d-48bc-86f5-8ef78124477f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c50ff4383b5c43788acb95c7c0a3d56f, user_id=5839cda8aa894d0093782221b42dbb50, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:00.910990 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9776544-ac6d-48bc-86f5-8ef78124477f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:00.956734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 932/1837] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:00 2022] PUT /identity/v3/projects/c50ff4383b5c43788acb95c7c0a3d56f/users/5839cda8aa894d0093782221b42dbb50/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:00.959081 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4190a62-ae6e-4681-8b08-9859ea8259ad None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:00.959533 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4190a62-ae6e-4681-8b08-9859ea8259ad None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:00.960255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4190a62-ae6e-4681-8b08-9859ea8259ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:00.969462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 906/1838] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:53:00.978007 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd1f91-31df-4389-950b-2e47c7c8ca9b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:00.978612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd1f91-31df-4389-950b-2e47c7c8ca9b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:00.979033 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd1f91-31df-4389-950b-2e47c7c8ca9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:00.987718 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c4190a62-ae6e-4681-8b08-9859ea8259ad None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:00.989255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c4190a62-ae6e-4681-8b08-9859ea8259ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:01.030048 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-efbd1f91-31df-4389-950b-2e47c7c8ca9b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:01.031903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-efbd1f91-31df-4389-950b-2e47c7c8ca9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:01.108485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 933/1839] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:01.111164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:01.132144 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 907/1840] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:01.136303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4190a62-ae6e-4681-8b08-9859ea8259ad None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:01.148057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4190a62-ae6e-4681-8b08-9859ea8259ad None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:01.154570 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ea593-600a-49fe-a0f3-95a2f0624827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:01.155192 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ea593-600a-49fe-a0f3-95a2f0624827 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:01.155485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ea593-600a-49fe-a0f3-95a2f0624827 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:01.155796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ea593-600a-49fe-a0f3-95a2f0624827 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:01.156493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ea593-600a-49fe-a0f3-95a2f0624827 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:01.156874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ea593-600a-49fe-a0f3-95a2f0624827 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:01.158725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ea593-600a-49fe-a0f3-95a2f0624827 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:01.175103 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 934/1841] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:53:01.179413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd1f91-31df-4389-950b-2e47c7c8ca9b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:01.193150 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd1f91-31df-4389-950b-2e47c7c8ca9b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:01.216484 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9210da3-76f4-4cec-b4ce-6f830b834327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:01.218332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9210da3-76f4-4cec-b4ce-6f830b834327 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:01.218974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9210da3-76f4-4cec-b4ce-6f830b834327 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:01.222416 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ea593-600a-49fe-a0f3-95a2f0624827 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:01.222977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9210da3-76f4-4cec-b4ce-6f830b834327 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:01.225758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c29ea593-600a-49fe-a0f3-95a2f0624827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:01.237779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9210da3-76f4-4cec-b4ce-6f830b834327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:01.242593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9210da3-76f4-4cec-b4ce-6f830b834327 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:01.298828 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:01.333606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 908/1842] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:01 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:01.337741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9210da3-76f4-4cec-b4ce-6f830b834327 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:01.359554 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ea593-600a-49fe-a0f3-95a2f0624827 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:01.425786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0e4fb-1a7e-4234-9bbc-668ecc611b41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:01.426932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0e4fb-1a7e-4234-9bbc-668ecc611b41 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:01.427550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0e4fb-1a7e-4234-9bbc-668ecc611b41 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:01.428200 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0e4fb-1a7e-4234-9bbc-668ecc611b41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:01.438383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9210da3-76f4-4cec-b4ce-6f830b834327 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:01.442835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0e4fb-1a7e-4234-9bbc-668ecc611b41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:01.444817 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0e4fb-1a7e-4234-9bbc-668ecc611b41 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:01.446831 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b943ba-48d7-4b6f-831d-264bf7c904f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:01.447465 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b943ba-48d7-4b6f-831d-264bf7c904f3 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:01.447747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b943ba-48d7-4b6f-831d-264bf7c904f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:01.448098 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b943ba-48d7-4b6f-831d-264bf7c904f3 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:01.448729 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b943ba-48d7-4b6f-831d-264bf7c904f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:01.451236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b943ba-48d7-4b6f-831d-264bf7c904f3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:01.457688 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 935/1843] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:01.475985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 909/1844] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:01 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:01.480575 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0e4fb-1a7e-4234-9bbc-668ecc611b41 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:01.502309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0e4fb-1a7e-4234-9bbc-668ecc611b41 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:01.509796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7eb70b-401a-441b-9e2b-3dede442b121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:01.510500 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7eb70b-401a-441b-9e2b-3dede442b121 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:01.510781 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7eb70b-401a-441b-9e2b-3dede442b121 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:01.511062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7eb70b-401a-441b-9e2b-3dede442b121 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:01.511724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7eb70b-401a-441b-9e2b-3dede442b121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:01.513669 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7eb70b-401a-441b-9e2b-3dede442b121 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:01.529016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 936/1845] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:01.533450 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b943ba-48d7-4b6f-831d-264bf7c904f3 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:01.546468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b943ba-48d7-4b6f-831d-264bf7c904f3 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:01.556965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e34a3d-cd19-4423-b675-bda555f33196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:01.558048 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e34a3d-cd19-4423-b675-bda555f33196 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:01.558386 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e34a3d-cd19-4423-b675-bda555f33196 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:01.558628 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e34a3d-cd19-4423-b675-bda555f33196 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:01.559755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e34a3d-cd19-4423-b675-bda555f33196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:01.561603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e34a3d-cd19-4423-b675-bda555f33196 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:01.578208 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 910/1846] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:01.579215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7eb70b-401a-441b-9e2b-3dede442b121 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:01.597623 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7eb70b-401a-441b-9e2b-3dede442b121 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:01.607013 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7cba9-4adb-48f0-8db0-5513534603a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:01.608929 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7cba9-4adb-48f0-8db0-5513534603a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:01.609280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7cba9-4adb-48f0-8db0-5513534603a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:01.609642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7cba9-4adb-48f0-8db0-5513534603a9 admin admin] PATH_INFO: `/v3/projects/da254dd28fba4f5b9cc6d98930b24496/users/064d007cd9da4852bb49fbbb5de97b87/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:01.639969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7cba9-4adb-48f0-8db0-5513534603a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da254dd28fba4f5b9cc6d98930b24496, user_id=064d007cd9da4852bb49fbbb5de97b87, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:01.642944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7cba9-4adb-48f0-8db0-5513534603a9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:01.671813 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 937/1847] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:01 2022] PUT /identity/v3/projects/da254dd28fba4f5b9cc6d98930b24496/users/064d007cd9da4852bb49fbbb5de97b87/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:01.677306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2e91f-fe21-4145-bac9-47d936f08cc7 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:01.677743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2e91f-fe21-4145-bac9-47d936f08cc7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:01.678056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2e91f-fe21-4145-bac9-47d936f08cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:01.730789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-46c2e91f-fe21-4145-bac9-47d936f08cc7 None None] MFA Rules not processed for user `064d007cd9da4852bb49fbbb5de97b87`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:01.732392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-46c2e91f-fe21-4145-bac9-47d936f08cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:01.827161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 911/1848] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:01.828461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7cba9-4adb-48f0-8db0-5513534603a9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:01.936563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7cba9-4adb-48f0-8db0-5513534603a9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:01.944342 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae159a-7599-4148-afdb-e7a774e0ce70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:01.944973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae159a-7599-4148-afdb-e7a774e0ce70 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:01.945915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae159a-7599-4148-afdb-e7a774e0ce70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:01.946203 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae159a-7599-4148-afdb-e7a774e0ce70 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:01.946864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae159a-7599-4148-afdb-e7a774e0ce70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:01.948896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ae159a-7599-4148-afdb-e7a774e0ce70 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:01.980994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 938/1849] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:01 2022] POST /identity/v3/projects => generated 387 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:53:01.984912 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2e91f-fe21-4145-bac9-47d936f08cc7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:01.997838 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2e91f-fe21-4145-bac9-47d936f08cc7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:02.044860 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:02.092169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc98b16-2384-49d9-a59d-137241ef620e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:02.093379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc98b16-2384-49d9-a59d-137241ef620e admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:02.093379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc98b16-2384-49d9-a59d-137241ef620e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:02.093977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc98b16-2384-49d9-a59d-137241ef620e admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:02.095138 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc98b16-2384-49d9-a59d-137241ef620e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:02.097714 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc98b16-2384-49d9-a59d-137241ef620e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:02.158096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 912/1850] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:01 2022] POST /identity/v3/users => generated 300 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:02.159402 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae159a-7599-4148-afdb-e7a774e0ce70 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:02.173175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae159a-7599-4148-afdb-e7a774e0ce70 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:02.179386 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691870f-5d08-4994-93ba-056a90a8f328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:02.180841 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691870f-5d08-4994-93ba-056a90a8f328 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:02.181379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691870f-5d08-4994-93ba-056a90a8f328 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:02.181615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691870f-5d08-4994-93ba-056a90a8f328 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:02.182575 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691870f-5d08-4994-93ba-056a90a8f328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:02.184289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691870f-5d08-4994-93ba-056a90a8f328 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:02.201533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc98b16-2384-49d9-a59d-137241ef620e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:02.211938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 939/1851] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:53:02.212683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc98b16-2384-49d9-a59d-137241ef620e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:02.220377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483292b-3f68-4ff3-9778-d00fe8141edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:02.221267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483292b-3f68-4ff3-9778-d00fe8141edb admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:02.221553 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483292b-3f68-4ff3-9778-d00fe8141edb admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:02.221855 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483292b-3f68-4ff3-9778-d00fe8141edb admin admin] PATH_INFO: `/v3/projects/42c1d02e6ab549c3af041cb6d1bbbf81/users/65ebfdbc38854ee494939f09961ea2aa/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:02.241247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483292b-3f68-4ff3-9778-d00fe8141edb admin admin] RBAC: Authorizing `identity:create_grant(project_id=42c1d02e6ab549c3af041cb6d1bbbf81, user_id=65ebfdbc38854ee494939f09961ea2aa, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:02.244578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e483292b-3f68-4ff3-9778-d00fe8141edb admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:02.263284 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691870f-5d08-4994-93ba-056a90a8f328 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:02.263503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 913/1852] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:02 2022] PUT /identity/v3/projects/42c1d02e6ab549c3af041cb6d1bbbf81/users/65ebfdbc38854ee494939f09961ea2aa/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:02.275987 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691870f-5d08-4994-93ba-056a90a8f328 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:02.329654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:02.452244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c004b-16b7-4dcf-bb70-2bf450d0e9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:02.453254 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c004b-16b7-4dcf-bb70-2bf450d0e9bb admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:02.453768 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c004b-16b7-4dcf-bb70-2bf450d0e9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:02.454271 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c004b-16b7-4dcf-bb70-2bf450d0e9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:02.455131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c004b-16b7-4dcf-bb70-2bf450d0e9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:02.457006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c004b-16b7-4dcf-bb70-2bf450d0e9bb admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:02.470745 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 940/1853] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:02.474946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483292b-3f68-4ff3-9778-d00fe8141edb admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:02.487441 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483292b-3f68-4ff3-9778-d00fe8141edb admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:02.493592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc905b7-db49-4063-a278-3e49e5b0947a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:02.494277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc905b7-db49-4063-a278-3e49e5b0947a admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:02.494554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc905b7-db49-4063-a278-3e49e5b0947a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:02.494848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc905b7-db49-4063-a278-3e49e5b0947a admin admin] PATH_INFO: `/v3/projects/42c1d02e6ab549c3af041cb6d1bbbf81/users/65ebfdbc38854ee494939f09961ea2aa/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:02.496923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc905b7-db49-4063-a278-3e49e5b0947a admin admin] RBAC: Authorizing `identity:create_grant(project_id=42c1d02e6ab549c3af041cb6d1bbbf81, user_id=65ebfdbc38854ee494939f09961ea2aa, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:02.499810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc905b7-db49-4063-a278-3e49e5b0947a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:02.522033 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 914/1854] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:02 2022] PUT /identity/v3/projects/42c1d02e6ab549c3af041cb6d1bbbf81/users/65ebfdbc38854ee494939f09961ea2aa/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:02.534003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4272049-068f-4a82-87bd-5765bc3d8fdd None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:02.534273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4272049-068f-4a82-87bd-5765bc3d8fdd None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:02.534987 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4272049-068f-4a82-87bd-5765bc3d8fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:02.591293 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b4272049-068f-4a82-87bd-5765bc3d8fdd None None] MFA Rules not processed for user `65ebfdbc38854ee494939f09961ea2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:02.593358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b4272049-068f-4a82-87bd-5765bc3d8fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:02.666982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc905b7-db49-4063-a278-3e49e5b0947a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:02.671410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-edc905b7-db49-4063-a278-3e49e5b0947a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:02.767857 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 941/1855] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:02.776182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4272049-068f-4a82-87bd-5765bc3d8fdd None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:02.777756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b4272049-068f-4a82-87bd-5765bc3d8fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:02.818058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc905b7-db49-4063-a278-3e49e5b0947a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:02.825935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7193e2d5-6e03-4e4c-8738-5bfe7862d6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:02.826985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193e2d5-6e03-4e4c-8738-5bfe7862d6fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:02.827419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193e2d5-6e03-4e4c-8738-5bfe7862d6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:02.827897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7193e2d5-6e03-4e4c-8738-5bfe7862d6fc admin admin] PATH_INFO: `/v3/projects/7d848eb6c172430bb3c1412b782da093` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:02.838456 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193e2d5-6e03-4e4c-8738-5bfe7862d6fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d848eb6c172430bb3c1412b782da093)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:02.841575 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7193e2d5-6e03-4e4c-8738-5bfe7862d6fc admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:02.915103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7193e2d5-6e03-4e4c-8738-5bfe7862d6fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d848eb6c172430bb3c1412b782da093', 'request_id': 'req-7193e2d5-6e03-4e4c-8738-5bfe7862d6fc'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:02.925986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7193e2d5-6e03-4e4c-8738-5bfe7862d6fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d848eb6c172430bb3c1412b782da093', 'request_id': 'req-7193e2d5-6e03-4e4c-8738-5bfe7862d6fc'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:02.936606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7193e2d5-6e03-4e4c-8738-5bfe7862d6fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d848eb6c172430bb3c1412b782da093', 'request_id': 'req-7193e2d5-6e03-4e4c-8738-5bfe7862d6fc'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:02.941700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4272049-068f-4a82-87bd-5765bc3d8fdd None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:02.976900 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7193e2d5-6e03-4e4c-8738-5bfe7862d6fc admin admin] The token cache is being invalidate because project 7d848eb6c172430bb3c1412b782da093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:02.978207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7193e2d5-6e03-4e4c-8738-5bfe7862d6fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:02.979731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-7193e2d5-6e03-4e4c-8738-5bfe7862d6fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:02.982219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 915/1856] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:02 2022] DELETE /identity/v3/projects/7d848eb6c172430bb3c1412b782da093 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:02.988253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea118e4-c66f-40e8-b68a-ab5899b8b78b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:02.988540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea118e4-c66f-40e8-b68a-ab5899b8b78b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:02.988819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea118e4-c66f-40e8-b68a-ab5899b8b78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:03.051775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86a616-4497-4f68-b901-cb7e925ed29e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:03.052705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86a616-4497-4f68-b901-cb7e925ed29e service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:03.053165 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86a616-4497-4f68-b901-cb7e925ed29e service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:03.053566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86a616-4497-4f68-b901-cb7e925ed29e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:03.056461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ee86a616-4497-4f68-b901-cb7e925ed29e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:03.064523 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0ea118e4-c66f-40e8-b68a-ab5899b8b78b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:03.067622 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0ea118e4-c66f-40e8-b68a-ab5899b8b78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:03.079780 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:03.116173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:03.197978 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86a616-4497-4f68-b901-cb7e925ed29e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:03.201050 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86a616-4497-4f68-b901-cb7e925ed29e service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:03.222744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 942/1857] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:53:03.240539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 916/1858] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:03.250610 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef692d63-b046-446b-973e-7444acfb7aea None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:03.251306 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef692d63-b046-446b-973e-7444acfb7aea None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:03.251593 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef692d63-b046-446b-973e-7444acfb7aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:03.317271 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ef692d63-b046-446b-973e-7444acfb7aea None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:03.318651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ef692d63-b046-446b-973e-7444acfb7aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:03.335521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 943/1859] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:03.338476 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea118e4-c66f-40e8-b68a-ab5899b8b78b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:03.351280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef692d63-b046-446b-973e-7444acfb7aea None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:03.355125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ef692d63-b046-446b-973e-7444acfb7aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:03.364020 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea118e4-c66f-40e8-b68a-ab5899b8b78b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:03.377729 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be0fa96-cd22-4833-9782-f6129bc738bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:03.378776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0fa96-cd22-4833-9782-f6129bc738bb admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:03.380284 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0fa96-cd22-4833-9782-f6129bc738bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:03.380670 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be0fa96-cd22-4833-9782-f6129bc738bb admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:03.381940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0fa96-cd22-4833-9782-f6129bc738bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:03.384593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0fa96-cd22-4833-9782-f6129bc738bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:03.389152 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be0fa96-cd22-4833-9782-f6129bc738bb admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:03.390321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef692d63-b046-446b-973e-7444acfb7aea None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:03.404523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e91af-25e1-41cd-951a-0ed78dc037c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:03.407675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e91af-25e1-41cd-951a-0ed78dc037c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:03.408088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e91af-25e1-41cd-951a-0ed78dc037c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:03.408466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e91af-25e1-41cd-951a-0ed78dc037c7 admin admin] PATH_INFO: `/v3/users/064d007cd9da4852bb49fbbb5de97b87` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:03.410671 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e91af-25e1-41cd-951a-0ed78dc037c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=064d007cd9da4852bb49fbbb5de97b87)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:03.416678 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4e91af-25e1-41cd-951a-0ed78dc037c7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:03.422902 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 917/1860] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:53:03.426460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9a49e-11c1-43ba-bb08-9e1a457235c8 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:03.427020 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9a49e-11c1-43ba-bb08-9e1a457235c8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:03.428229 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9a49e-11c1-43ba-bb08-9e1a457235c8 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:03.431528 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 918/1861] 10.222.0.60 () {54 vars in 921 bytes} [Mon Sep 19 14:53:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:53:03.435955 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9a49e-11c1-43ba-bb08-9e1a457235c8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:03.463018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9a49e-11c1-43ba-bb08-9e1a457235c8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:03.469268 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71b12d-f9b6-4b19-b30d-b5acc3b7857d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:03.471139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71b12d-f9b6-4b19-b30d-b5acc3b7857d admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:03.471139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71b12d-f9b6-4b19-b30d-b5acc3b7857d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:03.471318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71b12d-f9b6-4b19-b30d-b5acc3b7857d admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:03.472056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71b12d-f9b6-4b19-b30d-b5acc3b7857d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:03.474307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71b12d-f9b6-4b19-b30d-b5acc3b7857d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:03.546747 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ea4e91af-25e1-41cd-951a-0ed78dc037c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '064d007cd9da4852bb49fbbb5de97b87'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:03.562238 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ea4e91af-25e1-41cd-951a-0ed78dc037c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '064d007cd9da4852bb49fbbb5de97b87'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:03.573634 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 919/1862] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:03 2022] POST /identity/v3/projects => generated 367 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:03.578274 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b265e5-213e-43d4-b7e9-e02ade73d21d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:03.591920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b265e5-213e-43d4-b7e9-e02ade73d21d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:03.592919 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b265e5-213e-43d4-b7e9-e02ade73d21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:03.595985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ea4e91af-25e1-41cd-951a-0ed78dc037c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '064d007cd9da4852bb49fbbb5de97b87'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:03.618916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ea4e91af-25e1-41cd-951a-0ed78dc037c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '064d007cd9da4852bb49fbbb5de97b87'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:03.619679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ea4e91af-25e1-41cd-951a-0ed78dc037c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '064d007cd9da4852bb49fbbb5de97b87'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:03.628562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-25b265e5-213e-43d4-b7e9-e02ade73d21d None None] MFA Rules not processed for user `59b5d2fd362b417c9b9be75f88a667ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:03.630150 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-25b265e5-213e-43d4-b7e9-e02ade73d21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:03.634869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 944/1863] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:03 2022] DELETE /identity/v3/users/064d007cd9da4852bb49fbbb5de97b87 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:03.635921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e91af-25e1-41cd-951a-0ed78dc037c7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:03.640839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ea4e91af-25e1-41cd-951a-0ed78dc037c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:03.758618 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 920/1864] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Sep 19 14:53:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:03.760563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd84ca6-0d84-4bf8-b327-1fa6b1fec42d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:03.761639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd84ca6-0d84-4bf8-b327-1fa6b1fec42d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:03.762475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd84ca6-0d84-4bf8-b327-1fa6b1fec42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:03.793613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-bcd84ca6-0d84-4bf8-b327-1fa6b1fec42d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:03.795242 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bcd84ca6-0d84-4bf8-b327-1fa6b1fec42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:03.818362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e91af-25e1-41cd-951a-0ed78dc037c7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:03.831161 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111c3de-216e-4cce-a3c8-dc604f014852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:03.834001 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111c3de-216e-4cce-a3c8-dc604f014852 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:03.834659 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111c3de-216e-4cce-a3c8-dc604f014852 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:03.835925 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111c3de-216e-4cce-a3c8-dc604f014852 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:03.839891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111c3de-216e-4cce-a3c8-dc604f014852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:03.845202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111c3de-216e-4cce-a3c8-dc604f014852 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:03.878832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 945/1865] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:03 2022] POST /identity/v3/users => generated 284 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:03.880737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111c3de-216e-4cce-a3c8-dc604f014852 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:03.886991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d111c3de-216e-4cce-a3c8-dc604f014852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:03.895074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 921/1866] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:03.898503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd84ca6-0d84-4bf8-b327-1fa6b1fec42d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:03.901965 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:03.909784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd84ca6-0d84-4bf8-b327-1fa6b1fec42d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:03.917157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d54857-6e39-47c5-a59d-e5fe929962ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:03.917487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111c3de-216e-4cce-a3c8-dc604f014852 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:03.918094 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d54857-6e39-47c5-a59d-e5fe929962ba admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:03.918577 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d54857-6e39-47c5-a59d-e5fe929962ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:03.919133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d54857-6e39-47c5-a59d-e5fe929962ba admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:03.919789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d54857-6e39-47c5-a59d-e5fe929962ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:03.921863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d54857-6e39-47c5-a59d-e5fe929962ba admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:03.924558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388b9d4-6c6f-4e0a-ac57-2ad8cb5dea68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:03.926609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388b9d4-6c6f-4e0a-ac57-2ad8cb5dea68 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:03.926907 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388b9d4-6c6f-4e0a-ac57-2ad8cb5dea68 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:03.927227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388b9d4-6c6f-4e0a-ac57-2ad8cb5dea68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:03.934770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 922/1867] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:03.938693 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d54857-6e39-47c5-a59d-e5fe929962ba admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:03.943294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388b9d4-6c6f-4e0a-ac57-2ad8cb5dea68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:03.945186 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388b9d4-6c6f-4e0a-ac57-2ad8cb5dea68 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:03.951121 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:03.958577 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d54857-6e39-47c5-a59d-e5fe929962ba admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:03.959010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 946/1868] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:03.968193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388b9d4-6c6f-4e0a-ac57-2ad8cb5dea68 service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:03.970270 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f388b9d4-6c6f-4e0a-ac57-2ad8cb5dea68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:03.975303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2df761-470b-45a2-aa31-d7f330c0bd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:03.977421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2df761-470b-45a2-aa31-d7f330c0bd6e admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:03.978950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2df761-470b-45a2-aa31-d7f330c0bd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:03.978950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2df761-470b-45a2-aa31-d7f330c0bd6e admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:03.979388 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2df761-470b-45a2-aa31-d7f330c0bd6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:03.985549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2df761-470b-45a2-aa31-d7f330c0bd6e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:04.001432 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 923/1869] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:04.002560 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2df761-470b-45a2-aa31-d7f330c0bd6e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:04.026952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2df761-470b-45a2-aa31-d7f330c0bd6e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:04.038742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842c3a6-e413-43d5-8662-9559968704ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:04.039712 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842c3a6-e413-43d5-8662-9559968704ac admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:04.041308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842c3a6-e413-43d5-8662-9559968704ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:04.041308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842c3a6-e413-43d5-8662-9559968704ac admin admin] PATH_INFO: `/v3/projects/87a4dd5accdf413a834126ef3b0074e4/users/66f604bccf35435ea21a0bf3c56acea5/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:04.082151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842c3a6-e413-43d5-8662-9559968704ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=87a4dd5accdf413a834126ef3b0074e4, user_id=66f604bccf35435ea21a0bf3c56acea5, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:04.090940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8842c3a6-e413-43d5-8662-9559968704ac admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:04.098363 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388b9d4-6c6f-4e0a-ac57-2ad8cb5dea68 service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:04.104344 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d57b1-150d-4e0c-8992-f6da8d4b8cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:04.105131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d57b1-150d-4e0c-8992-f6da8d4b8cdd service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:04.105495 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d57b1-150d-4e0c-8992-f6da8d4b8cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:04.105840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5d57b1-150d-4e0c-8992-f6da8d4b8cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:04.116599 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 924/1870] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:04 2022] PUT /identity/v3/projects/87a4dd5accdf413a834126ef3b0074e4/users/66f604bccf35435ea21a0bf3c56acea5/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:04.124929 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d57b1-150d-4e0c-8992-f6da8d4b8cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:04.127694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c9bd7-cc1b-49b8-9f94-6ba192ae135b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:04.128037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c9bd7-cc1b-49b8-9f94-6ba192ae135b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:04.128693 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c9bd7-cc1b-49b8-9f94-6ba192ae135b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:04.129377 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5d57b1-150d-4e0c-8992-f6da8d4b8cdd service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:04.151969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 947/1871] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:53:04.162897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ea9c9bd7-cc1b-49b8-9f94-6ba192ae135b None None] MFA Rules not processed for user `66f604bccf35435ea21a0bf3c56acea5`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:04.164404 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ea9c9bd7-cc1b-49b8-9f94-6ba192ae135b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:04.250440 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 925/1872] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:04.255254 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d57b1-150d-4e0c-8992-f6da8d4b8cdd service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:04.266322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5d57b1-150d-4e0c-8992-f6da8d4b8cdd service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:04.316698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d8eee-af90-44e2-844a-49c058237604 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:04.316970 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d8eee-af90-44e2-844a-49c058237604 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:04.317255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d8eee-af90-44e2-844a-49c058237604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:04.378614 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f01d8eee-af90-44e2-844a-49c058237604 None None] MFA Rules not processed for user `5839cda8aa894d0093782221b42dbb50`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:04.379890 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f01d8eee-af90-44e2-844a-49c058237604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:04.413665 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71475d07-fa8e-4968-a3e9-a31f8c91b175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:04.415403 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71475d07-fa8e-4968-a3e9-a31f8c91b175 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:04.415817 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71475d07-fa8e-4968-a3e9-a31f8c91b175 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:04.416248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71475d07-fa8e-4968-a3e9-a31f8c91b175 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:04.417232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71475d07-fa8e-4968-a3e9-a31f8c91b175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:04.424757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71475d07-fa8e-4968-a3e9-a31f8c91b175 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:04.481854 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 948/1873] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:04 2022] POST /identity/v3/projects => generated 367 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:04.485869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71475d07-fa8e-4968-a3e9-a31f8c91b175 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:04.511526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 926/1874] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:53:04.516156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d8eee-af90-44e2-844a-49c058237604 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:04.518870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f01d8eee-af90-44e2-844a-49c058237604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:04.535598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d8eee-af90-44e2-844a-49c058237604 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:04.631076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:04.653454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71475d07-fa8e-4968-a3e9-a31f8c91b175 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:04.669742 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4c4ca-5dd9-4484-b676-76c4ae6c4ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:04.670361 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4c4ca-5dd9-4484-b676-76c4ae6c4ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:04.670642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4c4ca-5dd9-4484-b676-76c4ae6c4ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:04.670915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4c4ca-5dd9-4484-b676-76c4ae6c4ea9 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:04.671550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4c4ca-5dd9-4484-b676-76c4ae6c4ea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:04.676707 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4c4ca-5dd9-4484-b676-76c4ae6c4ea9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:04.680315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce07f54-32af-4fa3-ab3f-380f84126042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:04.681947 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce07f54-32af-4fa3-ab3f-380f84126042 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:04.682381 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce07f54-32af-4fa3-ab3f-380f84126042 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:04.682786 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce07f54-32af-4fa3-ab3f-380f84126042 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:04.683633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce07f54-32af-4fa3-ab3f-380f84126042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:04.686233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce07f54-32af-4fa3-ab3f-380f84126042 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:04.729296 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 949/1875] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:04 2022] POST /identity/v3/users => generated 290 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:04.733271 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4c4ca-5dd9-4484-b676-76c4ae6c4ea9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:04.737356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 927/1876] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:04 2022] POST /identity/v3/projects => generated 371 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:53:04.739157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce07f54-32af-4fa3-ab3f-380f84126042 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:04.747215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4c4ca-5dd9-4484-b676-76c4ae6c4ea9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:04.762224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:04.892976 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3e56b-29e6-4d8c-a80e-808a16dcecfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:04.893611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3e56b-29e6-4d8c-a80e-808a16dcecfe admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:04.893947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3e56b-29e6-4d8c-a80e-808a16dcecfe admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:04.894218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3e56b-29e6-4d8c-a80e-808a16dcecfe admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:04.894862 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3e56b-29e6-4d8c-a80e-808a16dcecfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:04.896573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3e56b-29e6-4d8c-a80e-808a16dcecfe admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:04.911876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 950/1877] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:04.915333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3e56b-29e6-4d8c-a80e-808a16dcecfe admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:04.932626 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3e56b-29e6-4d8c-a80e-808a16dcecfe admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:04.938360 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1da77-d778-4849-a9b5-18e1cdf0ad7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:04.939680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1da77-d778-4849-a9b5-18e1cdf0ad7a admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:04.940277 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1da77-d778-4849-a9b5-18e1cdf0ad7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:04.940701 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1da77-d778-4849-a9b5-18e1cdf0ad7a admin admin] PATH_INFO: `/v3/projects/e7a4acdf599744789ba4d2f61317aea3/users/3dfb20cda211403286b372b46cc030e7/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:04.946819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce07f54-32af-4fa3-ab3f-380f84126042 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:04.968904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88546e0-1449-4fec-b022-cca34b60802d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:04.969653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88546e0-1449-4fec-b022-cca34b60802d admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:04.969936 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88546e0-1449-4fec-b022-cca34b60802d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:04.970241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88546e0-1449-4fec-b022-cca34b60802d admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:04.972297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88546e0-1449-4fec-b022-cca34b60802d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:04.975766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88546e0-1449-4fec-b022-cca34b60802d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:04.983951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1da77-d778-4849-a9b5-18e1cdf0ad7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a4acdf599744789ba4d2f61317aea3, user_id=3dfb20cda211403286b372b46cc030e7, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:04.988805 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1da77-d778-4849-a9b5-18e1cdf0ad7a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:05.009767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 928/1878] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:04 2022] POST /identity/v3/users => generated 292 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:05.012243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88546e0-1449-4fec-b022-cca34b60802d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:05.014237 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d88546e0-1449-4fec-b022-cca34b60802d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:05.015628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 951/1879] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:04 2022] PUT /identity/v3/projects/e7a4acdf599744789ba4d2f61317aea3/users/3dfb20cda211403286b372b46cc030e7/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:05.016269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1da77-d778-4849-a9b5-18e1cdf0ad7a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:05.018860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c0a1da77-d778-4849-a9b5-18e1cdf0ad7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:05.031305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88546e0-1449-4fec-b022-cca34b60802d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:05.114471 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1da77-d778-4849-a9b5-18e1cdf0ad7a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:05.119496 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bb230-f586-4a29-b303-684c7381ad72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:05.119698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407e42a-cb73-4cb1-9de8-5e32be0380b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:05.120223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bb230-f586-4a29-b303-684c7381ad72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:05.120358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407e42a-cb73-4cb1-9de8-5e32be0380b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:05.120553 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bb230-f586-4a29-b303-684c7381ad72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:05.120742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407e42a-cb73-4cb1-9de8-5e32be0380b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:05.120848 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bb230-f586-4a29-b303-684c7381ad72 admin admin] PATH_INFO: `/v3/users/17d659700c0245e1929496c71b0635d6` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:05.121021 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407e42a-cb73-4cb1-9de8-5e32be0380b5 admin admin] PATH_INFO: `/v3/projects/da254dd28fba4f5b9cc6d98930b24496` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:05.122115 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753bb230-f586-4a29-b303-684c7381ad72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17d659700c0245e1929496c71b0635d6)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:05.122250 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407e42a-cb73-4cb1-9de8-5e32be0380b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da254dd28fba4f5b9cc6d98930b24496)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:05.124085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753bb230-f586-4a29-b303-684c7381ad72 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:05.124297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407e42a-cb73-4cb1-9de8-5e32be0380b5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:05.193529 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e407e42a-cb73-4cb1-9de8-5e32be0380b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da254dd28fba4f5b9cc6d98930b24496', 'request_id': 'req-e407e42a-cb73-4cb1-9de8-5e32be0380b5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:05.213974 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-753bb230-f586-4a29-b303-684c7381ad72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17d659700c0245e1929496c71b0635d6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:05.221596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e407e42a-cb73-4cb1-9de8-5e32be0380b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da254dd28fba4f5b9cc6d98930b24496', 'request_id': 'req-e407e42a-cb73-4cb1-9de8-5e32be0380b5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:05.223458 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-753bb230-f586-4a29-b303-684c7381ad72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17d659700c0245e1929496c71b0635d6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:05.234270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e407e42a-cb73-4cb1-9de8-5e32be0380b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da254dd28fba4f5b9cc6d98930b24496', 'request_id': 'req-e407e42a-cb73-4cb1-9de8-5e32be0380b5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:05.243982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:05.244195 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-753bb230-f586-4a29-b303-684c7381ad72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17d659700c0245e1929496c71b0635d6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:05.256913 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-753bb230-f586-4a29-b303-684c7381ad72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17d659700c0245e1929496c71b0635d6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:05.258811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-753bb230-f586-4a29-b303-684c7381ad72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17d659700c0245e1929496c71b0635d6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:05.259270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e407e42a-cb73-4cb1-9de8-5e32be0380b5 admin admin] The token cache is being invalidate because project da254dd28fba4f5b9cc6d98930b24496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:05.260054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e407e42a-cb73-4cb1-9de8-5e32be0380b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:05.260964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e407e42a-cb73-4cb1-9de8-5e32be0380b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:05.263725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:05.265258 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 929/1880] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:05 2022] DELETE /identity/v3/projects/da254dd28fba4f5b9cc6d98930b24496 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:05.266668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407e42a-cb73-4cb1-9de8-5e32be0380b5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:05.272523 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e407e42a-cb73-4cb1-9de8-5e32be0380b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:05.282550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 952/1881] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:05 2022] DELETE /identity/v3/users/17d659700c0245e1929496c71b0635d6 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:05.282550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bb230-f586-4a29-b303-684c7381ad72 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:05.286017 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-753bb230-f586-4a29-b303-684c7381ad72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:05.473599 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407e42a-cb73-4cb1-9de8-5e32be0380b5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:05.481559 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2d970-13b5-4b32-8ef7-6ee9b2c10ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:05.482220 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2d970-13b5-4b32-8ef7-6ee9b2c10ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:05.482733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2d970-13b5-4b32-8ef7-6ee9b2c10ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:05.483229 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2d970-13b5-4b32-8ef7-6ee9b2c10ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:05.484909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da2d970-13b5-4b32-8ef7-6ee9b2c10ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:05.487065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bb230-f586-4a29-b303-684c7381ad72 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:05.487483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da2d970-13b5-4b32-8ef7-6ee9b2c10ee6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:05.502718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 930/1882] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:05.502718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2d970-13b5-4b32-8ef7-6ee9b2c10ee6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:05.503445 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2fbdf-1128-4ba0-88d3-c87f7dd1beb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:05.504319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2fbdf-1128-4ba0-88d3-c87f7dd1beb4 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:05.504754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2fbdf-1128-4ba0-88d3-c87f7dd1beb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:05.505112 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7da2d970-13b5-4b32-8ef7-6ee9b2c10ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:05.505365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2fbdf-1128-4ba0-88d3-c87f7dd1beb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:05.506402 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2fbdf-1128-4ba0-88d3-c87f7dd1beb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:05.508779 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2fbdf-1128-4ba0-88d3-c87f7dd1beb4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:05.522333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 953/1883] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:05.524091 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb09796-32a0-4607-8c7d-46f63a2c94e0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:05.524526 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb09796-32a0-4607-8c7d-46f63a2c94e0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:05.524948 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb09796-32a0-4607-8c7d-46f63a2c94e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:05.537254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2d970-13b5-4b32-8ef7-6ee9b2c10ee6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:05.544136 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77323806-d871-49d8-892a-eab8be45c9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:05.544889 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77323806-d871-49d8-892a-eab8be45c9c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:05.545245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77323806-d871-49d8-892a-eab8be45c9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:05.546322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77323806-d871-49d8-892a-eab8be45c9c3 admin admin] PATH_INFO: `/v3/users/65ebfdbc38854ee494939f09961ea2aa` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:05.547817 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77323806-d871-49d8-892a-eab8be45c9c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65ebfdbc38854ee494939f09961ea2aa)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:05.549880 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77323806-d871-49d8-892a-eab8be45c9c3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:05.552627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-bbb09796-32a0-4607-8c7d-46f63a2c94e0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:05.554921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bbb09796-32a0-4607-8c7d-46f63a2c94e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:05.578225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 954/1884] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:05.582581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb09796-32a0-4607-8c7d-46f63a2c94e0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:05.596328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb09796-32a0-4607-8c7d-46f63a2c94e0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:05.602835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875eed6-f66c-4e36-9ce7-04e229171003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:05.603589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875eed6-f66c-4e36-9ce7-04e229171003 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:05.603903 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875eed6-f66c-4e36-9ce7-04e229171003 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:05.604217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875eed6-f66c-4e36-9ce7-04e229171003 admin admin] PATH_INFO: `/v3/projects/63a2ceed7faa45fba9c6fbf4f03c2d9e/users/06c9a34f6052478f9ad58e90b3b53f42/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:05.623041 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875eed6-f66c-4e36-9ce7-04e229171003 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a2ceed7faa45fba9c6fbf4f03c2d9e, user_id=06c9a34f6052478f9ad58e90b3b53f42, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:05.626561 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7875eed6-f66c-4e36-9ce7-04e229171003 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:05.627934 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-77323806-d871-49d8-892a-eab8be45c9c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65ebfdbc38854ee494939f09961ea2aa'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:05.641927 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-77323806-d871-49d8-892a-eab8be45c9c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65ebfdbc38854ee494939f09961ea2aa'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:05.652070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 955/1885] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:05 2022] PUT /identity/v3/projects/63a2ceed7faa45fba9c6fbf4f03c2d9e/users/06c9a34f6052478f9ad58e90b3b53f42/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:05.653111 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875eed6-f66c-4e36-9ce7-04e229171003 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:05.656865 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7875eed6-f66c-4e36-9ce7-04e229171003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:05.665206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-77323806-d871-49d8-892a-eab8be45c9c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65ebfdbc38854ee494939f09961ea2aa'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:05.676335 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-77323806-d871-49d8-892a-eab8be45c9c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65ebfdbc38854ee494939f09961ea2aa'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:05.678484 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-77323806-d871-49d8-892a-eab8be45c9c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65ebfdbc38854ee494939f09961ea2aa'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:05.692782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 931/1886] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:05 2022] DELETE /identity/v3/users/65ebfdbc38854ee494939f09961ea2aa => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:05.695107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77323806-d871-49d8-892a-eab8be45c9c3 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:05.698510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-77323806-d871-49d8-892a-eab8be45c9c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:05.789646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7875eed6-f66c-4e36-9ce7-04e229171003 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:05.791006 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77323806-d871-49d8-892a-eab8be45c9c3 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:05.796487 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee943299-36e1-4a5f-840b-fb4c5e157888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:05.797253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee943299-36e1-4a5f-840b-fb4c5e157888 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:05.797630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee943299-36e1-4a5f-840b-fb4c5e157888 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:05.797983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee943299-36e1-4a5f-840b-fb4c5e157888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:05.800096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ee943299-36e1-4a5f-840b-fb4c5e157888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:05.820920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:05.889181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfed9a-f24a-4154-bfea-1a027e233567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:05.889520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfed9a-f24a-4154-bfea-1a027e233567 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:05.891975 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfed9a-f24a-4154-bfea-1a027e233567 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:05.892855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfed9a-f24a-4154-bfea-1a027e233567 admin admin] PATH_INFO: `/v3/projects/e7a4acdf599744789ba4d2f61317aea3/users/3dfb20cda211403286b372b46cc030e7/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:05.898886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdfed9a-f24a-4154-bfea-1a027e233567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a4acdf599744789ba4d2f61317aea3, user_id=3dfb20cda211403286b372b46cc030e7, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:05.908267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdfed9a-f24a-4154-bfea-1a027e233567 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:05.932158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 956/1887] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:05 2022] PUT /identity/v3/projects/e7a4acdf599744789ba4d2f61317aea3/users/3dfb20cda211403286b372b46cc030e7/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:05.933975 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfed9a-f24a-4154-bfea-1a027e233567 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:05.936111 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee943299-36e1-4a5f-840b-fb4c5e157888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:05.937482 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9fdfed9a-f24a-4154-bfea-1a027e233567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:05.937761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee943299-36e1-4a5f-840b-fb4c5e157888 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:05.950616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 932/1888] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:05.952018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f020033-db4a-4b6d-9549-7f34d1807b14 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:05.952497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f020033-db4a-4b6d-9549-7f34d1807b14 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:05.952785 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f020033-db4a-4b6d-9549-7f34d1807b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:05.957247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:05.995373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-4f020033-db4a-4b6d-9549-7f34d1807b14 None None] MFA Rules not processed for user `3dfb20cda211403286b372b46cc030e7`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:05.996863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4f020033-db4a-4b6d-9549-7f34d1807b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:06.052448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfed9a-f24a-4154-bfea-1a027e233567 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:06.059267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2aa8f-636d-4ffd-a217-7c8e4426b4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:06.061150 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2aa8f-636d-4ffd-a217-7c8e4426b4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:06.061439 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2aa8f-636d-4ffd-a217-7c8e4426b4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:06.061720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2aa8f-636d-4ffd-a217-7c8e4426b4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:06.062552 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f2aa8f-636d-4ffd-a217-7c8e4426b4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:06.064952 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f2aa8f-636d-4ffd-a217-7c8e4426b4d6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:06.079887 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 957/1889] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:06.085497 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2aa8f-636d-4ffd-a217-7c8e4426b4d6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:06.110469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2aa8f-636d-4ffd-a217-7c8e4426b4d6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:06.122884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc40052-a2e9-465a-a8a9-7bf8700a0d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:06.123944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc40052-a2e9-465a-a8a9-7bf8700a0d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:06.124536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc40052-a2e9-465a-a8a9-7bf8700a0d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:06.125154 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc40052-a2e9-465a-a8a9-7bf8700a0d89 admin admin] PATH_INFO: `/v3/projects/63a2ceed7faa45fba9c6fbf4f03c2d9e/users/06c9a34f6052478f9ad58e90b3b53f42/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:06.128984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc40052-a2e9-465a-a8a9-7bf8700a0d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63a2ceed7faa45fba9c6fbf4f03c2d9e, user_id=06c9a34f6052478f9ad58e90b3b53f42, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:06.133286 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc40052-a2e9-465a-a8a9-7bf8700a0d89 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:06.149814 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 958/1890] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:06 2022] PUT /identity/v3/projects/63a2ceed7faa45fba9c6fbf4f03c2d9e/users/06c9a34f6052478f9ad58e90b3b53f42/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:06.151998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 933/1891] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:05 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:06.158175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc40052-a2e9-465a-a8a9-7bf8700a0d89 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:06.160402 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ddc40052-a2e9-465a-a8a9-7bf8700a0d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:06.230346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc40052-a2e9-465a-a8a9-7bf8700a0d89 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:06.234740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff5dc5-11e7-41af-a177-65380f1a15c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:06.235365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff5dc5-11e7-41af-a177-65380f1a15c2 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:06.235647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff5dc5-11e7-41af-a177-65380f1a15c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:06.235968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff5dc5-11e7-41af-a177-65380f1a15c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:06.332943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff5dc5-11e7-41af-a177-65380f1a15c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:06.334714 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff5dc5-11e7-41af-a177-65380f1a15c2 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:06.347965 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 959/1892] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:06 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:06.633637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f020033-db4a-4b6d-9549-7f34d1807b14 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:06.635289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4f020033-db4a-4b6d-9549-7f34d1807b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:06.655507 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f020033-db4a-4b6d-9549-7f34d1807b14 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:06.824316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946387d-ef20-4237-94f1-73ca9acd48a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:06.825658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946387d-ef20-4237-94f1-73ca9acd48a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:06.827204 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946387d-ef20-4237-94f1-73ca9acd48a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:06.827543 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c946387d-ef20-4237-94f1-73ca9acd48a3 admin admin] PATH_INFO: `/v3/projects/42c1d02e6ab549c3af041cb6d1bbbf81` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:06.829380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946387d-ef20-4237-94f1-73ca9acd48a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42c1d02e6ab549c3af041cb6d1bbbf81)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:06.831407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c946387d-ef20-4237-94f1-73ca9acd48a3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:06.921001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:06.936388 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c946387d-ef20-4237-94f1-73ca9acd48a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42c1d02e6ab549c3af041cb6d1bbbf81', 'request_id': 'req-c946387d-ef20-4237-94f1-73ca9acd48a3'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:06.947512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c946387d-ef20-4237-94f1-73ca9acd48a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42c1d02e6ab549c3af041cb6d1bbbf81', 'request_id': 'req-c946387d-ef20-4237-94f1-73ca9acd48a3'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:06.959005 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c946387d-ef20-4237-94f1-73ca9acd48a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42c1d02e6ab549c3af041cb6d1bbbf81', 'request_id': 'req-c946387d-ef20-4237-94f1-73ca9acd48a3'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:06.978285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c946387d-ef20-4237-94f1-73ca9acd48a3 admin admin] The token cache is being invalidate because project 42c1d02e6ab549c3af041cb6d1bbbf81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:06.978285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c946387d-ef20-4237-94f1-73ca9acd48a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:06.978724 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c946387d-ef20-4237-94f1-73ca9acd48a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:06.980836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 934/1893] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:06 2022] DELETE /identity/v3/projects/42c1d02e6ab549c3af041cb6d1bbbf81 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:06.986261 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe171b-eb12-4156-bb9f-ab926b110069 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:06.986261 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe171b-eb12-4156-bb9f-ab926b110069 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:06.986441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe171b-eb12-4156-bb9f-ab926b110069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:07.010224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2dfe171b-eb12-4156-bb9f-ab926b110069 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:07.012081 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2dfe171b-eb12-4156-bb9f-ab926b110069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:07.037931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946387d-ef20-4237-94f1-73ca9acd48a3 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:07.040046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c946387d-ef20-4237-94f1-73ca9acd48a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:07.105567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:07.179972 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 960/1894] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:07.185144 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5d13e-371d-4a4b-a1f7-4fe8f32dcde8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:07.186069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5d13e-371d-4a4b-a1f7-4fe8f32dcde8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:07.200073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5d13e-371d-4a4b-a1f7-4fe8f32dcde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:07.211556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c946387d-ef20-4237-94f1-73ca9acd48a3 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:07.232397 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b8e5d13e-371d-4a4b-a1f7-4fe8f32dcde8 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:07.233782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b8e5d13e-371d-4a4b-a1f7-4fe8f32dcde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:07.236780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63a377-cc47-4a57-ac3b-f67ee420eb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:07.239097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63a377-cc47-4a57-ac3b-f67ee420eb17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:07.240886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63a377-cc47-4a57-ac3b-f67ee420eb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:07.241312 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63a377-cc47-4a57-ac3b-f67ee420eb17 admin admin] PATH_INFO: `/v3/projects/8e4e7b1217614c50a78d547810a6b238` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:07.243905 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 961/1895] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:07.248076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5d13e-371d-4a4b-a1f7-4fe8f32dcde8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:07.249150 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63a377-cc47-4a57-ac3b-f67ee420eb17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e4e7b1217614c50a78d547810a6b238)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:07.254158 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63a377-cc47-4a57-ac3b-f67ee420eb17 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:07.261215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5d13e-371d-4a4b-a1f7-4fe8f32dcde8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:07.267774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cd205-fc54-48cc-8242-99557bac2a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:07.269235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cd205-fc54-48cc-8242-99557bac2a00 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:07.269235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cd205-fc54-48cc-8242-99557bac2a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:07.269613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cd205-fc54-48cc-8242-99557bac2a00 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:07.270763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cd205-fc54-48cc-8242-99557bac2a00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:07.271564 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cd205-fc54-48cc-8242-99557bac2a00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:07.274561 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cd205-fc54-48cc-8242-99557bac2a00 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:07.297021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 962/1896] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:53:07.298062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cd205-fc54-48cc-8242-99557bac2a00 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:07.311361 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cd205-fc54-48cc-8242-99557bac2a00 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:07.319445 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2569e-f1f6-4ca5-87d4-b41e67322c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:07.321801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2569e-f1f6-4ca5-87d4-b41e67322c68 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:07.322098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2569e-f1f6-4ca5-87d4-b41e67322c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:07.322378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2569e-f1f6-4ca5-87d4-b41e67322c68 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:07.323158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2569e-f1f6-4ca5-87d4-b41e67322c68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:07.328418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2569e-f1f6-4ca5-87d4-b41e67322c68 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:07.371988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 963/1897] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:07 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:07.376153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2569e-f1f6-4ca5-87d4-b41e67322c68 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:07.380804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5a63a377-cc47-4a57-ac3b-f67ee420eb17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e4e7b1217614c50a78d547810a6b238', 'request_id': 'req-5a63a377-cc47-4a57-ac3b-f67ee420eb17'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:07.394541 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5a63a377-cc47-4a57-ac3b-f67ee420eb17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e4e7b1217614c50a78d547810a6b238', 'request_id': 'req-5a63a377-cc47-4a57-ac3b-f67ee420eb17'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:07.420318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5a63a377-cc47-4a57-ac3b-f67ee420eb17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e4e7b1217614c50a78d547810a6b238', 'request_id': 'req-5a63a377-cc47-4a57-ac3b-f67ee420eb17'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:07.436052 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5a63a377-cc47-4a57-ac3b-f67ee420eb17 admin admin] The token cache is being invalidate because project 8e4e7b1217614c50a78d547810a6b238 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:07.436697 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5a63a377-cc47-4a57-ac3b-f67ee420eb17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:07.437571 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5a63a377-cc47-4a57-ac3b-f67ee420eb17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:07.442958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 935/1898] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:07 2022] DELETE /identity/v3/projects/8e4e7b1217614c50a78d547810a6b238 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:07.444141 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63a377-cc47-4a57-ac3b-f67ee420eb17 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:07.446009 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5a63a377-cc47-4a57-ac3b-f67ee420eb17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:07.493798 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2569e-f1f6-4ca5-87d4-b41e67322c68 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:07.509872 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10929923-35b1-46b0-9c82-3efdb70933a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:07.510510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10929923-35b1-46b0-9c82-3efdb70933a9 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:07.511291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10929923-35b1-46b0-9c82-3efdb70933a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:07.511291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10929923-35b1-46b0-9c82-3efdb70933a9 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:07.511699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10929923-35b1-46b0-9c82-3efdb70933a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:07.524135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10929923-35b1-46b0-9c82-3efdb70933a9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:07.564152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 964/1899] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:07 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:07.568257 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10929923-35b1-46b0-9c82-3efdb70933a9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:07.571590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-10929923-35b1-46b0-9c82-3efdb70933a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:07.599273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10929923-35b1-46b0-9c82-3efdb70933a9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:07.600994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:07.608490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63a377-cc47-4a57-ac3b-f67ee420eb17 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:07.610173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894ac05-bf9a-4be1-bf33-eefcc28b3d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:07.610811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894ac05-bf9a-4be1-bf33-eefcc28b3d0f admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:07.611943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894ac05-bf9a-4be1-bf33-eefcc28b3d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:07.612226 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894ac05-bf9a-4be1-bf33-eefcc28b3d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:07.613059 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894ac05-bf9a-4be1-bf33-eefcc28b3d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:07.615925 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894ac05-bf9a-4be1-bf33-eefcc28b3d0f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:07.623267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19eae9d-6d3d-4739-864f-95ee80004991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:07.626711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eae9d-6d3d-4739-864f-95ee80004991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:07.626711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eae9d-6d3d-4739-864f-95ee80004991 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:07.626711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19eae9d-6d3d-4739-864f-95ee80004991 admin admin] PATH_INFO: `/v3/users/7099a63ac97b4fcdbe66f94f353f0a1d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:07.628202 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19eae9d-6d3d-4739-864f-95ee80004991 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7099a63ac97b4fcdbe66f94f353f0a1d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:07.632896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19eae9d-6d3d-4739-864f-95ee80004991 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:07.640247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 965/1900] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:53:07.644528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894ac05-bf9a-4be1-bf33-eefcc28b3d0f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:07.663632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894ac05-bf9a-4be1-bf33-eefcc28b3d0f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:07.669551 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b96883-a7d7-4c65-9f31-c123e68d16d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:07.670152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b96883-a7d7-4c65-9f31-c123e68d16d6 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:07.670493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b96883-a7d7-4c65-9f31-c123e68d16d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:07.670965 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b96883-a7d7-4c65-9f31-c123e68d16d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:07.671546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b96883-a7d7-4c65-9f31-c123e68d16d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:07.673830 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b96883-a7d7-4c65-9f31-c123e68d16d6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:07.694822 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 966/1901] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:07.695981 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b96883-a7d7-4c65-9f31-c123e68d16d6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:07.728143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b96883-a7d7-4c65-9f31-c123e68d16d6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:07.733900 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b162ed9-651b-40a0-96c7-649669f10806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:07.734993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b162ed9-651b-40a0-96c7-649669f10806 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:07.735567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b162ed9-651b-40a0-96c7-649669f10806 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:07.736155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b162ed9-651b-40a0-96c7-649669f10806 admin admin] PATH_INFO: `/v3/projects/b81e9fa9f9b64d049655f13f3d523a43/users/907a4d1342b748c6ad18611955d9a18e/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:07.760886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d19eae9d-6d3d-4739-864f-95ee80004991 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7099a63ac97b4fcdbe66f94f353f0a1d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:07.772283 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b162ed9-651b-40a0-96c7-649669f10806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b81e9fa9f9b64d049655f13f3d523a43, user_id=907a4d1342b748c6ad18611955d9a18e, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:07.792951 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d19eae9d-6d3d-4739-864f-95ee80004991 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7099a63ac97b4fcdbe66f94f353f0a1d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:07.796115 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b162ed9-651b-40a0-96c7-649669f10806 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:07.832122 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d19eae9d-6d3d-4739-864f-95ee80004991 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7099a63ac97b4fcdbe66f94f353f0a1d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:07.841463 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 967/1902] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:07 2022] PUT /identity/v3/projects/b81e9fa9f9b64d049655f13f3d523a43/users/907a4d1342b748c6ad18611955d9a18e/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:07.843482 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c989bf-d79a-400f-9d52-0d569388c89a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:07.843735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c989bf-d79a-400f-9d52-0d569388c89a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:07.844046 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c989bf-d79a-400f-9d52-0d569388c89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:07.849544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d19eae9d-6d3d-4739-864f-95ee80004991 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7099a63ac97b4fcdbe66f94f353f0a1d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:07.852397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d19eae9d-6d3d-4739-864f-95ee80004991 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7099a63ac97b4fcdbe66f94f353f0a1d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:07.877627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 936/1903] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:07 2022] DELETE /identity/v3/users/7099a63ac97b4fcdbe66f94f353f0a1d => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:07.880889 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7a36c-a07f-49b7-82f8-a911d28cfd14 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:07.883425 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7a36c-a07f-49b7-82f8-a911d28cfd14 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:07.883814 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a7a36c-a07f-49b7-82f8-a911d28cfd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:07.888761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b6c989bf-d79a-400f-9d52-0d569388c89a None None] MFA Rules not processed for user `06c9a34f6052478f9ad58e90b3b53f42`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:07.896405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b6c989bf-d79a-400f-9d52-0d569388c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:07.939483 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:07.959734 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-57a7a36c-a07f-49b7-82f8-a911d28cfd14 None None] MFA Rules not processed for user `907a4d1342b748c6ad18611955d9a18e`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:07.962916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-57a7a36c-a07f-49b7-82f8-a911d28cfd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:08.051550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 968/1904] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:08.099743 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c989bf-d79a-400f-9d52-0d569388c89a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:08.102055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b6c989bf-d79a-400f-9d52-0d569388c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:08.103198 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 937/1905] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:08.129310 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7a36c-a07f-49b7-82f8-a911d28cfd14 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:08.131791 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-57a7a36c-a07f-49b7-82f8-a911d28cfd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:08.220093 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a7a36c-a07f-49b7-82f8-a911d28cfd14 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:08.224518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392197f2-9d18-4a2b-a97e-7569c67b6272 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:08.225467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392197f2-9d18-4a2b-a97e-7569c67b6272 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:08.225804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392197f2-9d18-4a2b-a97e-7569c67b6272 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:08.227080 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392197f2-9d18-4a2b-a97e-7569c67b6272 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:08.239982 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c989bf-d79a-400f-9d52-0d569388c89a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:08.247862 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e4908-1283-4545-9621-0c29d6b96c23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:08.248500 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e4908-1283-4545-9621-0c29d6b96c23 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:08.249190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392197f2-9d18-4a2b-a97e-7569c67b6272 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:08.250606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e4908-1283-4545-9621-0c29d6b96c23 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:08.250884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e4908-1283-4545-9621-0c29d6b96c23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:08.254416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392197f2-9d18-4a2b-a97e-7569c67b6272 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:08.258702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3a6e4908-1283-4545-9621-0c29d6b96c23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:08.271803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 938/1906] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:53:08.367528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e4908-1283-4545-9621-0c29d6b96c23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:08.369401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e4908-1283-4545-9621-0c29d6b96c23 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:08.382874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 969/1907] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:53:09.392743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392197f2-9d18-4a2b-a97e-7569c67b6272 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:09.394941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-392197f2-9d18-4a2b-a97e-7569c67b6272 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:09.407320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:09.413069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e4908-1283-4545-9621-0c29d6b96c23 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:09.414783 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3a6e4908-1283-4545-9621-0c29d6b96c23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:09.495353 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392197f2-9d18-4a2b-a97e-7569c67b6272 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:09.499645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e4908-1283-4545-9621-0c29d6b96c23 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:09.500592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2523933-614a-4185-b82b-10a4dd686373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:09.501413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2523933-614a-4185-b82b-10a4dd686373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:09.501695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2523933-614a-4185-b82b-10a4dd686373 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:09.501974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2523933-614a-4185-b82b-10a4dd686373 admin admin] PATH_INFO: `/v3/projects/ad6335dc2fb64d03b66469e58f296822` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:09.503150 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2523933-614a-4185-b82b-10a4dd686373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad6335dc2fb64d03b66469e58f296822)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:09.505111 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2523933-614a-4185-b82b-10a4dd686373 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:09.506592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e49295-a914-47f0-a9ca-fc11b29b326c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:09.507344 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e49295-a914-47f0-a9ca-fc11b29b326c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:09.507801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e49295-a914-47f0-a9ca-fc11b29b326c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:09.508206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e49295-a914-47f0-a9ca-fc11b29b326c admin admin] PATH_INFO: `/v3/users/66f604bccf35435ea21a0bf3c56acea5` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:09.509421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e49295-a914-47f0-a9ca-fc11b29b326c admin admin] RBAC: Authorizing `identity:delete_user(user_id=66f604bccf35435ea21a0bf3c56acea5)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:09.511423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e49295-a914-47f0-a9ca-fc11b29b326c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:09.539632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:09.562792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e2523933-614a-4185-b82b-10a4dd686373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad6335dc2fb64d03b66469e58f296822', 'request_id': 'req-e2523933-614a-4185-b82b-10a4dd686373'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:09.568943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e1e49295-a914-47f0-a9ca-fc11b29b326c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66f604bccf35435ea21a0bf3c56acea5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:09.570492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e2523933-614a-4185-b82b-10a4dd686373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad6335dc2fb64d03b66469e58f296822', 'request_id': 'req-e2523933-614a-4185-b82b-10a4dd686373'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:09.576945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e1e49295-a914-47f0-a9ca-fc11b29b326c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66f604bccf35435ea21a0bf3c56acea5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:09.581777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e2523933-614a-4185-b82b-10a4dd686373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad6335dc2fb64d03b66469e58f296822', 'request_id': 'req-e2523933-614a-4185-b82b-10a4dd686373'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:09.590406 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e1e49295-a914-47f0-a9ca-fc11b29b326c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66f604bccf35435ea21a0bf3c56acea5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:09.594584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e2523933-614a-4185-b82b-10a4dd686373 admin admin] The token cache is being invalidate because project ad6335dc2fb64d03b66469e58f296822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:09.594943 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e2523933-614a-4185-b82b-10a4dd686373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:09.595682 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e2523933-614a-4185-b82b-10a4dd686373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:09.597268 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 939/1908] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:09 2022] DELETE /identity/v3/projects/ad6335dc2fb64d03b66469e58f296822 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:09.598946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e1e49295-a914-47f0-a9ca-fc11b29b326c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66f604bccf35435ea21a0bf3c56acea5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:09.599818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e1e49295-a914-47f0-a9ca-fc11b29b326c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66f604bccf35435ea21a0bf3c56acea5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:09.603601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eb758-a0bf-4d8f-81c8-f5e22cf2ae6f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:09.604023 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eb758-a0bf-4d8f-81c8-f5e22cf2ae6f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:09.604369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5eb758-a0bf-4d8f-81c8-f5e22cf2ae6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:09.607256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 970/1909] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:09 2022] DELETE /identity/v3/users/66f604bccf35435ea21a0bf3c56acea5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:09.611376 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea360cc-fafb-4f7b-a948-961adbb2a88f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:09.611664 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea360cc-fafb-4f7b-a948-961adbb2a88f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:09.612037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea360cc-fafb-4f7b-a948-961adbb2a88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:09.628941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-bd5eb758-a0bf-4d8f-81c8-f5e22cf2ae6f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:09.630314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bd5eb758-a0bf-4d8f-81c8-f5e22cf2ae6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:09.635069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2ea360cc-fafb-4f7b-a948-961adbb2a88f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:09.636301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2ea360cc-fafb-4f7b-a948-961adbb2a88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:09.698146 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 940/1910] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:09.705762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42e00f-af0f-4ca9-91ea-249269e2de9a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:09.706130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42e00f-af0f-4ca9-91ea-249269e2de9a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:09.706480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42e00f-af0f-4ca9-91ea-249269e2de9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:09.710573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 971/1911] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:53:09.718915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea360cc-fafb-4f7b-a948-961adbb2a88f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:09.721211 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2ea360cc-fafb-4f7b-a948-961adbb2a88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:09.730708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ff42e00f-af0f-4ca9-91ea-249269e2de9a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:09.732613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ff42e00f-af0f-4ca9-91ea-249269e2de9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:09.742475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 941/1912] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:53:09.746517 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42e00f-af0f-4ca9-91ea-249269e2de9a None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:09.761045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42e00f-af0f-4ca9-91ea-249269e2de9a None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:09.767759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae197ec0-c98d-4b1d-992f-2b4774db55de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:09.768818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae197ec0-c98d-4b1d-992f-2b4774db55de admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:09.769159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae197ec0-c98d-4b1d-992f-2b4774db55de admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:09.769496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae197ec0-c98d-4b1d-992f-2b4774db55de admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:09.770214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae197ec0-c98d-4b1d-992f-2b4774db55de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:09.770953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae197ec0-c98d-4b1d-992f-2b4774db55de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:09.773246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae197ec0-c98d-4b1d-992f-2b4774db55de admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:09.785605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 942/1913] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:53:09.787443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea360cc-fafb-4f7b-a948-961adbb2a88f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:09.790777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae197ec0-c98d-4b1d-992f-2b4774db55de admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:09.793428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43a844-20df-4513-9f6a-d39a5c2ad915 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:09.794085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43a844-20df-4513-9f6a-d39a5c2ad915 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:09.794361 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43a844-20df-4513-9f6a-d39a5c2ad915 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:09.794633 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43a844-20df-4513-9f6a-d39a5c2ad915 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:09.801790 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae197ec0-c98d-4b1d-992f-2b4774db55de admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:09.805988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43a844-20df-4513-9f6a-d39a5c2ad915 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:09.807682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43a844-20df-4513-9f6a-d39a5c2ad915 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:09.808701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b93640-73a1-444d-83bf-59b2af64ee78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:09.809315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b93640-73a1-444d-83bf-59b2af64ee78 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:09.809597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b93640-73a1-444d-83bf-59b2af64ee78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:09.809926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b93640-73a1-444d-83bf-59b2af64ee78 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:09.810555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b93640-73a1-444d-83bf-59b2af64ee78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:09.812531 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b93640-73a1-444d-83bf-59b2af64ee78 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:09.817926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 972/1914] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:53:09.850343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:09.872224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 943/1915] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:09 2022] POST /identity/v3/projects => generated 363 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:53:09.873128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43a844-20df-4513-9f6a-d39a5c2ad915 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:09.885369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43a844-20df-4513-9f6a-d39a5c2ad915 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:09.997469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:10.038419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d947bcb-a958-456d-98ff-ac261814a591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:10.039427 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d947bcb-a958-456d-98ff-ac261814a591 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:10.039905 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d947bcb-a958-456d-98ff-ac261814a591 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:10.040188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d947bcb-a958-456d-98ff-ac261814a591 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:10.040884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d947bcb-a958-456d-98ff-ac261814a591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:10.046208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d947bcb-a958-456d-98ff-ac261814a591 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:10.125810 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 973/1916] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:09 2022] POST /identity/v3/users => generated 282 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:53:10.127888 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d947bcb-a958-456d-98ff-ac261814a591 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:10.128085 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b93640-73a1-444d-83bf-59b2af64ee78 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:10.129612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1d947bcb-a958-456d-98ff-ac261814a591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:10.168738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b93640-73a1-444d-83bf-59b2af64ee78 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:10.192499 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187205-4cfc-4d75-bd03-e8f8b756b554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:10.193100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187205-4cfc-4d75-bd03-e8f8b756b554 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:10.193459 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187205-4cfc-4d75-bd03-e8f8b756b554 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:10.193743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3187205-4cfc-4d75-bd03-e8f8b756b554 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:10.194736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3187205-4cfc-4d75-bd03-e8f8b756b554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:10.197629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3187205-4cfc-4d75-bd03-e8f8b756b554 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:10.211717 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 944/1917] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:10.216716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187205-4cfc-4d75-bd03-e8f8b756b554 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:10.228774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3187205-4cfc-4d75-bd03-e8f8b756b554 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:10.234922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d947bcb-a958-456d-98ff-ac261814a591 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:10.236699 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18456d9e-d1f4-43ca-a758-e43e8d6a0dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:10.237572 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18456d9e-d1f4-43ca-a758-e43e8d6a0dbe admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:10.238121 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18456d9e-d1f4-43ca-a758-e43e8d6a0dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:10.239164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18456d9e-d1f4-43ca-a758-e43e8d6a0dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:10.240886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18456d9e-d1f4-43ca-a758-e43e8d6a0dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:10.244950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18456d9e-d1f4-43ca-a758-e43e8d6a0dbe admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:10.249107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d959867-45f8-4de8-9532-65bbbccc4a53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:10.249864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d959867-45f8-4de8-9532-65bbbccc4a53 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:10.250216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d959867-45f8-4de8-9532-65bbbccc4a53 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:10.250566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d959867-45f8-4de8-9532-65bbbccc4a53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:10.253305 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8d959867-45f8-4de8-9532-65bbbccc4a53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:10.272676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 945/1918] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:10 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:10.274401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18456d9e-d1f4-43ca-a758-e43e8d6a0dbe admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:10.324498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18456d9e-d1f4-43ca-a758-e43e8d6a0dbe admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:10.340639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85239537-68df-4032-a501-4f2a63630516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:10.341628 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85239537-68df-4032-a501-4f2a63630516 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:10.341742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85239537-68df-4032-a501-4f2a63630516 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:10.343139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85239537-68df-4032-a501-4f2a63630516 admin admin] PATH_INFO: `/v3/projects/f4d74d98fbbe4a26ad74fff76a0a7c9c/users/bc3d3c8b185648ef8ab8528d924a5b6d/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:10.424140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85239537-68df-4032-a501-4f2a63630516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d74d98fbbe4a26ad74fff76a0a7c9c, user_id=bc3d3c8b185648ef8ab8528d924a5b6d, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:10.436771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85239537-68df-4032-a501-4f2a63630516 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:10.470348 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d959867-45f8-4de8-9532-65bbbccc4a53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:10.475174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 946/1919] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:10 2022] PUT /identity/v3/projects/f4d74d98fbbe4a26ad74fff76a0a7c9c/users/bc3d3c8b185648ef8ab8528d924a5b6d/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:10.476063 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85239537-68df-4032-a501-4f2a63630516 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:10.485418 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-85239537-68df-4032-a501-4f2a63630516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:10.485900 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d959867-45f8-4de8-9532-65bbbccc4a53 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:10.513976 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 974/1920] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:53:10.515626 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d462293-ab35-4b37-8a23-e42e2154d4f3 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:10.516049 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d462293-ab35-4b37-8a23-e42e2154d4f3 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:10.516433 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d462293-ab35-4b37-8a23-e42e2154d4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:10.558181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0d462293-ab35-4b37-8a23-e42e2154d4f3 None None] MFA Rules not processed for user `bc3d3c8b185648ef8ab8528d924a5b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:10.559620 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0d462293-ab35-4b37-8a23-e42e2154d4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:10.653865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85239537-68df-4032-a501-4f2a63630516 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:10.670771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef5926-b9ef-4073-8e94-a1ea990bd591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:10.671147 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef5926-b9ef-4073-8e94-a1ea990bd591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:10.671554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef5926-b9ef-4073-8e94-a1ea990bd591 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:10.672542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef5926-b9ef-4073-8e94-a1ea990bd591 admin admin] PATH_INFO: `/v3/projects/87a4dd5accdf413a834126ef3b0074e4` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:10.674547 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef5926-b9ef-4073-8e94-a1ea990bd591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87a4dd5accdf413a834126ef3b0074e4)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:10.677413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef5926-b9ef-4073-8e94-a1ea990bd591 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:10.718641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 975/1921] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:10.721459 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85335996-af39-449e-800b-559f5c6d7c3c None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:10.721815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85335996-af39-449e-800b-559f5c6d7c3c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:10.722197 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85335996-af39-449e-800b-559f5c6d7c3c None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:10.724057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 976/1922] 10.222.0.60 () {54 vars in 921 bytes} [Mon Sep 19 14:53:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:53:10.724996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85335996-af39-449e-800b-559f5c6d7c3c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:10.756030 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85335996-af39-449e-800b-559f5c6d7c3c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:10.787493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:10.789837 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6bef5926-b9ef-4073-8e94-a1ea990bd591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87a4dd5accdf413a834126ef3b0074e4', 'request_id': 'req-6bef5926-b9ef-4073-8e94-a1ea990bd591'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:10.799187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6bef5926-b9ef-4073-8e94-a1ea990bd591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87a4dd5accdf413a834126ef3b0074e4', 'request_id': 'req-6bef5926-b9ef-4073-8e94-a1ea990bd591'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:10.821646 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6bef5926-b9ef-4073-8e94-a1ea990bd591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87a4dd5accdf413a834126ef3b0074e4', 'request_id': 'req-6bef5926-b9ef-4073-8e94-a1ea990bd591'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:10.837524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6bef5926-b9ef-4073-8e94-a1ea990bd591 admin admin] The token cache is being invalidate because project 87a4dd5accdf413a834126ef3b0074e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:10.837822 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6bef5926-b9ef-4073-8e94-a1ea990bd591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:10.840665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6bef5926-b9ef-4073-8e94-a1ea990bd591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:10.844421 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 947/1923] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:10 2022] DELETE /identity/v3/projects/87a4dd5accdf413a834126ef3b0074e4 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:10.845899 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03945448-0953-4154-832e-f949610c88a2 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:10.846398 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03945448-0953-4154-832e-f949610c88a2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:10.846505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03945448-0953-4154-832e-f949610c88a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:10.889662 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-03945448-0953-4154-832e-f949610c88a2 None None] MFA Rules not processed for user `59b5d2fd362b417c9b9be75f88a667ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:10.891036 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-03945448-0953-4154-832e-f949610c88a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:10.926127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6ad13-f209-4ce8-99fa-a5835e23d432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:10.927365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6ad13-f209-4ce8-99fa-a5835e23d432 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:10.927846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6ad13-f209-4ce8-99fa-a5835e23d432 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:10.928322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6ad13-f209-4ce8-99fa-a5835e23d432 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:10.929140 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6ad13-f209-4ce8-99fa-a5835e23d432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:10.936086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6ad13-f209-4ce8-99fa-a5835e23d432 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:10.937459 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:10.993449 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 948/1924] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Sep 19 14:53:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:10.994151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03945448-0953-4154-832e-f949610c88a2 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:11.000382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-03945448-0953-4154-832e-f949610c88a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:11.006046 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 977/1925] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:10 2022] POST /identity/v3/projects => generated 363 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:11.010339 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6ad13-f209-4ce8-99fa-a5835e23d432 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:11.012925 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-95b6ad13-f209-4ce8-99fa-a5835e23d432 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:11.022572 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03945448-0953-4154-832e-f949610c88a2 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:11.106635 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6ad13-f209-4ce8-99fa-a5835e23d432 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:11.134319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a4620-94b5-441f-b3f3-f4b3ed0a4ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:11.134962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a4620-94b5-441f-b3f3-f4b3ed0a4ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:11.135248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a4620-94b5-441f-b3f3-f4b3ed0a4ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:11.135766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a4620-94b5-441f-b3f3-f4b3ed0a4ec6 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:11.136446 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a4620-94b5-441f-b3f3-f4b3ed0a4ec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:11.138552 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a4620-94b5-441f-b3f3-f4b3ed0a4ec6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:11.147157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ded0cc-2887-47f2-b456-a16ceedb3657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:11.148355 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ded0cc-2887-47f2-b456-a16ceedb3657 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:11.148355 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ded0cc-2887-47f2-b456-a16ceedb3657 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:11.148693 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ded0cc-2887-47f2-b456-a16ceedb3657 admin admin] PATH_INFO: `/v3/users/3dfb20cda211403286b372b46cc030e7` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:11.149878 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ded0cc-2887-47f2-b456-a16ceedb3657 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dfb20cda211403286b372b46cc030e7)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:11.151918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ded0cc-2887-47f2-b456-a16ceedb3657 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:11.168005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 978/1926] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:11 2022] POST /identity/v3/users => generated 288 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:53:11.169685 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a4620-94b5-441f-b3f3-f4b3ed0a4ec6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:11.172583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-812a4620-94b5-441f-b3f3-f4b3ed0a4ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:11.206856 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-09ded0cc-2887-47f2-b456-a16ceedb3657 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dfb20cda211403286b372b46cc030e7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:11.217665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-09ded0cc-2887-47f2-b456-a16ceedb3657 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dfb20cda211403286b372b46cc030e7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:11.231057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-09ded0cc-2887-47f2-b456-a16ceedb3657 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dfb20cda211403286b372b46cc030e7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:11.239298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-09ded0cc-2887-47f2-b456-a16ceedb3657 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dfb20cda211403286b372b46cc030e7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:11.240088 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-09ded0cc-2887-47f2-b456-a16ceedb3657 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dfb20cda211403286b372b46cc030e7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:11.246932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 949/1927] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:10 2022] DELETE /identity/v3/users/3dfb20cda211403286b372b46cc030e7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:11.247671 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ded0cc-2887-47f2-b456-a16ceedb3657 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:11.249028 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-09ded0cc-2887-47f2-b456-a16ceedb3657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:11.256993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a4620-94b5-441f-b3f3-f4b3ed0a4ec6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:11.293748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:11.333234 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:11.357449 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ded0cc-2887-47f2-b456-a16ceedb3657 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:11.363360 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602bb8b-1db7-4854-ac60-f783fc509f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:11.363999 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed385a-9ce5-493b-b191-1319fb83dda4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:11.364671 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602bb8b-1db7-4854-ac60-f783fc509f8c admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:11.364782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed385a-9ce5-493b-b191-1319fb83dda4 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:11.364961 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed385a-9ce5-493b-b191-1319fb83dda4 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:11.365242 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed385a-9ce5-493b-b191-1319fb83dda4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:11.365576 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602bb8b-1db7-4854-ac60-f783fc509f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:11.366185 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602bb8b-1db7-4854-ac60-f783fc509f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:11.367289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2fed385a-9ce5-493b-b191-1319fb83dda4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:11.368347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602bb8b-1db7-4854-ac60-f783fc509f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:11.370398 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602bb8b-1db7-4854-ac60-f783fc509f8c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:11.381181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed385a-9ce5-493b-b191-1319fb83dda4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:11.382837 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fed385a-9ce5-493b-b191-1319fb83dda4 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:11.384174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 950/1928] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:11.388917 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602bb8b-1db7-4854-ac60-f783fc509f8c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:11.395438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 979/1929] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:11.401567 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602bb8b-1db7-4854-ac60-f783fc509f8c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:11.406882 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31d869-9706-42d2-bd94-4cc56623f9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:11.407680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31d869-9706-42d2-bd94-4cc56623f9e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:11.407911 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31d869-9706-42d2-bd94-4cc56623f9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:11.408145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31d869-9706-42d2-bd94-4cc56623f9e4 admin admin] PATH_INFO: `/v3/projects/485f53bbf73a47a4aee4b8eeb6cef862/users/8f5c72768cd14148ae6743e82c59920c/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:11.426786 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31d869-9706-42d2-bd94-4cc56623f9e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=485f53bbf73a47a4aee4b8eeb6cef862, user_id=8f5c72768cd14148ae6743e82c59920c, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:11.429877 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31d869-9706-42d2-bd94-4cc56623f9e4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:11.447789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 951/1930] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:11 2022] PUT /identity/v3/projects/485f53bbf73a47a4aee4b8eeb6cef862/users/8f5c72768cd14148ae6743e82c59920c/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:11.449011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed385a-9ce5-493b-b191-1319fb83dda4 service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:11.468015 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed385a-9ce5-493b-b191-1319fb83dda4 service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:11.533649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b1949-d4db-4b1d-8350-bf54da37f84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:11.534426 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b1949-d4db-4b1d-8350-bf54da37f84e admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:11.534710 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b1949-d4db-4b1d-8350-bf54da37f84e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:11.534984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b1949-d4db-4b1d-8350-bf54da37f84e admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:11.536124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b1949-d4db-4b1d-8350-bf54da37f84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:11.537317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5b1949-d4db-4b1d-8350-bf54da37f84e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:11.554644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 980/1931] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:11.559350 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31d869-9706-42d2-bd94-4cc56623f9e4 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:11.572767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b1949-d4db-4b1d-8350-bf54da37f84e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:11.574327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31d869-9706-42d2-bd94-4cc56623f9e4 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:11.575680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bf5b1949-d4db-4b1d-8350-bf54da37f84e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:11.579481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6da72e-c786-41b4-9226-c9182a141b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:11.580924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6da72e-c786-41b4-9226-c9182a141b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:11.580924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6da72e-c786-41b4-9226-c9182a141b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:11.580924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6da72e-c786-41b4-9226-c9182a141b31 admin admin] PATH_INFO: `/v3/projects/485f53bbf73a47a4aee4b8eeb6cef862/users/8f5c72768cd14148ae6743e82c59920c/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:11.583086 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6da72e-c786-41b4-9226-c9182a141b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=485f53bbf73a47a4aee4b8eeb6cef862, user_id=8f5c72768cd14148ae6743e82c59920c, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:11.586558 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6da72e-c786-41b4-9226-c9182a141b31 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:11.598984 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 952/1932] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:11 2022] PUT /identity/v3/projects/485f53bbf73a47a4aee4b8eeb6cef862/users/8f5c72768cd14148ae6743e82c59920c/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:11.603535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b1949-d4db-4b1d-8350-bf54da37f84e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:11.612996 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb212fb-45ea-4ad4-80b0-f2fbf805e218 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:11.613276 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb212fb-45ea-4ad4-80b0-f2fbf805e218 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:11.613554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb212fb-45ea-4ad4-80b0-f2fbf805e218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:11.645097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-7bb212fb-45ea-4ad4-80b0-f2fbf805e218 None None] MFA Rules not processed for user `8f5c72768cd14148ae6743e82c59920c`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:11.646638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7bb212fb-45ea-4ad4-80b0-f2fbf805e218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:11.672402 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6765f3-19ea-4070-b378-a9efd5ef5814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:11.673045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6765f3-19ea-4070-b378-a9efd5ef5814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:11.673328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6765f3-19ea-4070-b378-a9efd5ef5814 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:11.673604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6765f3-19ea-4070-b378-a9efd5ef5814 admin admin] PATH_INFO: `/v3/projects/e7a4acdf599744789ba4d2f61317aea3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:11.674753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6765f3-19ea-4070-b378-a9efd5ef5814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7a4acdf599744789ba4d2f61317aea3)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:11.676695 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6765f3-19ea-4070-b378-a9efd5ef5814 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:11.721429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 953/1933] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:53:11.730855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ab6765f3-19ea-4070-b378-a9efd5ef5814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7a4acdf599744789ba4d2f61317aea3', 'request_id': 'req-ab6765f3-19ea-4070-b378-a9efd5ef5814'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:11.732085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:11.732881 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb212fb-45ea-4ad4-80b0-f2fbf805e218 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:11.734175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7bb212fb-45ea-4ad4-80b0-f2fbf805e218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:11.738296 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ab6765f3-19ea-4070-b378-a9efd5ef5814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7a4acdf599744789ba4d2f61317aea3', 'request_id': 'req-ab6765f3-19ea-4070-b378-a9efd5ef5814'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:11.744987 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ab6765f3-19ea-4070-b378-a9efd5ef5814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7a4acdf599744789ba4d2f61317aea3', 'request_id': 'req-ab6765f3-19ea-4070-b378-a9efd5ef5814'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:11.758470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ab6765f3-19ea-4070-b378-a9efd5ef5814 admin admin] The token cache is being invalidate because project e7a4acdf599744789ba4d2f61317aea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:11.758755 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ab6765f3-19ea-4070-b378-a9efd5ef5814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:11.759797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ab6765f3-19ea-4070-b378-a9efd5ef5814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:11.761373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 981/1934] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:11 2022] DELETE /identity/v3/projects/e7a4acdf599744789ba4d2f61317aea3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:11.767358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1959d-33b3-4752-a359-e80050ca6cc8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:11.767813 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1959d-33b3-4752-a359-e80050ca6cc8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:11.768124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1959d-33b3-4752-a359-e80050ca6cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:11.791108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9be1959d-33b3-4752-a359-e80050ca6cc8 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:11.791892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb212fb-45ea-4ad4-80b0-f2fbf805e218 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:11.792360 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9be1959d-33b3-4752-a359-e80050ca6cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:11.853668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66928d78-c262-47f4-ba27-cd0b73b8867c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:11.854371 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66928d78-c262-47f4-ba27-cd0b73b8867c service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:11.854658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66928d78-c262-47f4-ba27-cd0b73b8867c service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:11.854934 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66928d78-c262-47f4-ba27-cd0b73b8867c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:11.855874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 982/1935] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:11.857576 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-66928d78-c262-47f4-ba27-cd0b73b8867c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:11.865046 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79979b-cd9b-43fd-ae28-3ed675455119 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:11.865335 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79979b-cd9b-43fd-ae28-3ed675455119 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:11.865611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79979b-cd9b-43fd-ae28-3ed675455119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:11.866601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:11.893375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-fc79979b-cd9b-43fd-ae28-3ed675455119 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:11.894460 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fc79979b-cd9b-43fd-ae28-3ed675455119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:11.904409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 983/1936] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:53:11.909233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79979b-cd9b-43fd-ae28-3ed675455119 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:11.920534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79979b-cd9b-43fd-ae28-3ed675455119 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:11.926463 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff073c53-cf55-4ce8-8a65-cbf4e3227bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:11.927077 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff073c53-cf55-4ce8-8a65-cbf4e3227bda admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:11.927403 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff073c53-cf55-4ce8-8a65-cbf4e3227bda admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:11.927612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff073c53-cf55-4ce8-8a65-cbf4e3227bda admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:11.928279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff073c53-cf55-4ce8-8a65-cbf4e3227bda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:11.928577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff073c53-cf55-4ce8-8a65-cbf4e3227bda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:11.930796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff073c53-cf55-4ce8-8a65-cbf4e3227bda admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:11.935881 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66928d78-c262-47f4-ba27-cd0b73b8867c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:11.937361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66928d78-c262-47f4-ba27-cd0b73b8867c service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:11.945590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 984/1937] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:53:11.949946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff073c53-cf55-4ce8-8a65-cbf4e3227bda admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:11.951189 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 954/1938] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:11 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:53:11.960210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff073c53-cf55-4ce8-8a65-cbf4e3227bda admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:11.965396 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363519f-09a2-4220-801e-b0be8847bca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:11.966159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363519f-09a2-4220-801e-b0be8847bca5 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:11.966378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363519f-09a2-4220-801e-b0be8847bca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:11.966647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363519f-09a2-4220-801e-b0be8847bca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:11.968943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363519f-09a2-4220-801e-b0be8847bca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:11.969821 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363519f-09a2-4220-801e-b0be8847bca5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:12.013381 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 985/1939] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:11 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:53:12.014530 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66928d78-c262-47f4-ba27-cd0b73b8867c service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:12.027990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66928d78-c262-47f4-ba27-cd0b73b8867c service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:12.050024 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363519f-09a2-4220-801e-b0be8847bca5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:12.051853 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1363519f-09a2-4220-801e-b0be8847bca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:12.097064 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a45215-c732-4e5a-9bdf-e2da58c1e12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:12.097683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a45215-c732-4e5a-9bdf-e2da58c1e12b admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:12.098285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a45215-c732-4e5a-9bdf-e2da58c1e12b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:12.098540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a45215-c732-4e5a-9bdf-e2da58c1e12b admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:12.099181 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a45215-c732-4e5a-9bdf-e2da58c1e12b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:12.101711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a45215-c732-4e5a-9bdf-e2da58c1e12b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:12.118967 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363519f-09a2-4220-801e-b0be8847bca5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:12.123117 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074cb416-360c-49c9-9c25-98e3526db455 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:12.123745 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cb416-360c-49c9-9c25-98e3526db455 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:12.124063 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cb416-360c-49c9-9c25-98e3526db455 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:12.124349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074cb416-360c-49c9-9c25-98e3526db455 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:12.126438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-074cb416-360c-49c9-9c25-98e3526db455 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:12.127681 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 955/1940] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:12 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:12.131885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a45215-c732-4e5a-9bdf-e2da58c1e12b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:12.144864 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a45215-c732-4e5a-9bdf-e2da58c1e12b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:12.150913 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04945d21-386b-426c-a9c8-d789b14a2747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:12.151537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04945d21-386b-426c-a9c8-d789b14a2747 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:12.151842 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04945d21-386b-426c-a9c8-d789b14a2747 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:12.152277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04945d21-386b-426c-a9c8-d789b14a2747 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:12.152826 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04945d21-386b-426c-a9c8-d789b14a2747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:12.154447 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04945d21-386b-426c-a9c8-d789b14a2747 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:12.170581 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 956/1941] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:53:12.173974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04945d21-386b-426c-a9c8-d789b14a2747 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:12.190482 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04945d21-386b-426c-a9c8-d789b14a2747 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:12.192602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:12.203043 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb842a-e56c-4134-a2a2-e824b4898371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:12.203660 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb842a-e56c-4134-a2a2-e824b4898371 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:12.204213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb842a-e56c-4134-a2a2-e824b4898371 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:12.204213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb842a-e56c-4134-a2a2-e824b4898371 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:12.204866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb842a-e56c-4134-a2a2-e824b4898371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:12.208295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcb842a-e56c-4134-a2a2-e824b4898371 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:12.213309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074cb416-360c-49c9-9c25-98e3526db455 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:12.215624 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074cb416-360c-49c9-9c25-98e3526db455 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:12.222177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 957/1942] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:12.227380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb842a-e56c-4134-a2a2-e824b4898371 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:12.240132 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 986/1943] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:12.246088 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb842a-e56c-4134-a2a2-e824b4898371 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:12.254818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd0c92-a874-40b3-a976-c0c77ea1c167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:12.255965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd0c92-a874-40b3-a976-c0c77ea1c167 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:12.256427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd0c92-a874-40b3-a976-c0c77ea1c167 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:12.256878 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd0c92-a874-40b3-a976-c0c77ea1c167 admin admin] PATH_INFO: `/v3/projects/3609e51cd0b14d96985bfe990965d46f/users/b657ebf3a1a2464da98daa0b621baf81/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:12.278543 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd0c92-a874-40b3-a976-c0c77ea1c167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3609e51cd0b14d96985bfe990965d46f, user_id=b657ebf3a1a2464da98daa0b621baf81, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:12.281586 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd0c92-a874-40b3-a976-c0c77ea1c167 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:12.303957 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 958/1944] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:12 2022] PUT /identity/v3/projects/3609e51cd0b14d96985bfe990965d46f/users/b657ebf3a1a2464da98daa0b621baf81/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:12.312596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ff7fe-d01e-4557-9aaf-720827e88504 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:12.312894 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ff7fe-d01e-4557-9aaf-720827e88504 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:12.313170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ff7fe-d01e-4557-9aaf-720827e88504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:12.377130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-6d8ff7fe-d01e-4557-9aaf-720827e88504 None None] MFA Rules not processed for user `b657ebf3a1a2464da98daa0b621baf81`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:12.378255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6d8ff7fe-d01e-4557-9aaf-720827e88504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:12.451658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 987/1945] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:12.456166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd0c92-a874-40b3-a976-c0c77ea1c167 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:12.505111 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:12.520470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd0c92-a874-40b3-a976-c0c77ea1c167 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:12.525761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840ace2-0784-44aa-bede-42ffca924cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:12.526364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840ace2-0784-44aa-bede-42ffca924cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:12.526649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840ace2-0784-44aa-bede-42ffca924cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:12.526931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840ace2-0784-44aa-bede-42ffca924cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:12.527567 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840ace2-0784-44aa-bede-42ffca924cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:12.529524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840ace2-0784-44aa-bede-42ffca924cc6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:12.556694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 959/1946] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:12 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:12.560962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ff7fe-d01e-4557-9aaf-720827e88504 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:12.572575 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ff7fe-d01e-4557-9aaf-720827e88504 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:12.628142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7346d4-5f23-4adf-92d2-950ea4e05da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:12.628766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7346d4-5f23-4adf-92d2-950ea4e05da7 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:12.629053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7346d4-5f23-4adf-92d2-950ea4e05da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:12.629329 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7346d4-5f23-4adf-92d2-950ea4e05da7 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:12.630253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7346d4-5f23-4adf-92d2-950ea4e05da7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:12.631942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7346d4-5f23-4adf-92d2-950ea4e05da7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:12.656239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 988/1947] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:12 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:12.660404 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840ace2-0784-44aa-bede-42ffca924cc6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:12.674474 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840ace2-0784-44aa-bede-42ffca924cc6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:12.681940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182f50d-fafd-4dcf-9d6b-10469835ed9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:12.682947 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182f50d-fafd-4dcf-9d6b-10469835ed9f admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:12.683500 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182f50d-fafd-4dcf-9d6b-10469835ed9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:12.683974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182f50d-fafd-4dcf-9d6b-10469835ed9f admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:12.685004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182f50d-fafd-4dcf-9d6b-10469835ed9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:12.688011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182f50d-fafd-4dcf-9d6b-10469835ed9f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:12.701188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 960/1948] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:12.704997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7346d4-5f23-4adf-92d2-950ea4e05da7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:12.715873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7346d4-5f23-4adf-92d2-950ea4e05da7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:12.720668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5de5e-6619-44c9-973a-241dd272add1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:12.721285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5de5e-6619-44c9-973a-241dd272add1 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:12.721565 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5de5e-6619-44c9-973a-241dd272add1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:12.721839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5de5e-6619-44c9-973a-241dd272add1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:12.722512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5de5e-6619-44c9-973a-241dd272add1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:12.724143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5de5e-6619-44c9-973a-241dd272add1 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:12.735175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 989/1949] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:12.738772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182f50d-fafd-4dcf-9d6b-10469835ed9f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:12.749960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182f50d-fafd-4dcf-9d6b-10469835ed9f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:12.754863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e3816-dc15-46c2-89f3-fe3d6af9c049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:12.755869 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e3816-dc15-46c2-89f3-fe3d6af9c049 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:12.756103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e3816-dc15-46c2-89f3-fe3d6af9c049 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:12.756378 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78e3816-dc15-46c2-89f3-fe3d6af9c049 admin admin] PATH_INFO: `/v3/projects/e94139dff6ec4277be7eb5960b7ec4d9/users/8df9c6322740499f8f9816f544a723d7/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:12.773691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e3816-dc15-46c2-89f3-fe3d6af9c049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e94139dff6ec4277be7eb5960b7ec4d9, user_id=8df9c6322740499f8f9816f544a723d7, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:12.776457 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78e3816-dc15-46c2-89f3-fe3d6af9c049 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:12.789465 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 961/1950] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:12 2022] PUT /identity/v3/projects/e94139dff6ec4277be7eb5960b7ec4d9/users/8df9c6322740499f8f9816f544a723d7/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:12.800531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703abe1-92dd-4a9d-9b4c-ba523b133582 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:12.800850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703abe1-92dd-4a9d-9b4c-ba523b133582 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:12.801174 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703abe1-92dd-4a9d-9b4c-ba523b133582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:12.831617 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4703abe1-92dd-4a9d-9b4c-ba523b133582 None None] MFA Rules not processed for user `8df9c6322740499f8f9816f544a723d7`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:12.832859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4703abe1-92dd-4a9d-9b4c-ba523b133582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:12.882171 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:12.903656 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 990/1951] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:12.908163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e3816-dc15-46c2-89f3-fe3d6af9c049 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:12.969955 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78e3816-dc15-46c2-89f3-fe3d6af9c049 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:12.974586 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc11948-4d0f-4da5-8adb-067dee31269c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:12.975187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc11948-4d0f-4da5-8adb-067dee31269c admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:12.975471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc11948-4d0f-4da5-8adb-067dee31269c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:12.975802 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc11948-4d0f-4da5-8adb-067dee31269c admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:12.976483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc11948-4d0f-4da5-8adb-067dee31269c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:12.978374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc11948-4d0f-4da5-8adb-067dee31269c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:13.010118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 962/1952] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:12 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:13.011147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703abe1-92dd-4a9d-9b4c-ba523b133582 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:13.024322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703abe1-92dd-4a9d-9b4c-ba523b133582 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:13.104322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ec55c-47b5-492f-b969-263b7a88db04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:13.104858 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ec55c-47b5-492f-b969-263b7a88db04 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:13.105133 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ec55c-47b5-492f-b969-263b7a88db04 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:13.105404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ec55c-47b5-492f-b969-263b7a88db04 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:13.106046 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ec55c-47b5-492f-b969-263b7a88db04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:13.108108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ec55c-47b5-492f-b969-263b7a88db04 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:13.135572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 991/1953] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:13 2022] POST /identity/v3/users => generated 299 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:53:13.137314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc11948-4d0f-4da5-8adb-067dee31269c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:13.151526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc11948-4d0f-4da5-8adb-067dee31269c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:13.158490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a013d-1f07-40a5-b0f8-6ddf49bc3612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:13.159287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a013d-1f07-40a5-b0f8-6ddf49bc3612 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:13.159633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a013d-1f07-40a5-b0f8-6ddf49bc3612 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:13.159990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a013d-1f07-40a5-b0f8-6ddf49bc3612 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:13.160981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a013d-1f07-40a5-b0f8-6ddf49bc3612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:13.162607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a013d-1f07-40a5-b0f8-6ddf49bc3612 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:13.176823 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 963/1954] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:13.181984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ec55c-47b5-492f-b969-263b7a88db04 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:13.195890 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ec55c-47b5-492f-b969-263b7a88db04 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:13.200940 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d4ef8-f2b1-422a-964d-ba91fc73adad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:13.201614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d4ef8-f2b1-422a-964d-ba91fc73adad admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:13.201903 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d4ef8-f2b1-422a-964d-ba91fc73adad admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:13.202181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d4ef8-f2b1-422a-964d-ba91fc73adad admin admin] PATH_INFO: `/v3/projects/2ea87718cc274350bf823760c372804b/users/ac9d2c51bedf4b85b5c706bd6d5e30d2/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:13.223515 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d4ef8-f2b1-422a-964d-ba91fc73adad admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ea87718cc274350bf823760c372804b, user_id=ac9d2c51bedf4b85b5c706bd6d5e30d2, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:13.228031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d4ef8-f2b1-422a-964d-ba91fc73adad admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:13.246877 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 992/1955] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:13 2022] PUT /identity/v3/projects/2ea87718cc274350bf823760c372804b/users/ac9d2c51bedf4b85b5c706bd6d5e30d2/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:13.251766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a013d-1f07-40a5-b0f8-6ddf49bc3612 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:13.265356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a013d-1f07-40a5-b0f8-6ddf49bc3612 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:13.332503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17b2257-47f0-4fe7-915a-d86990d08fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:13.334744 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b2257-47f0-4fe7-915a-d86990d08fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:13.334744 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b2257-47f0-4fe7-915a-d86990d08fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:13.334744 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17b2257-47f0-4fe7-915a-d86990d08fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:13.340636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b2257-47f0-4fe7-915a-d86990d08fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:13.342378 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17b2257-47f0-4fe7-915a-d86990d08fb7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:13.360383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d4ef8-f2b1-422a-964d-ba91fc73adad admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:13.361446 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 964/1956] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:13.383040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d4ef8-f2b1-422a-964d-ba91fc73adad admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:13.398062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa26a7f-8b1e-40bf-87d4-95c50dc73747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:13.398802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa26a7f-8b1e-40bf-87d4-95c50dc73747 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:13.399080 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa26a7f-8b1e-40bf-87d4-95c50dc73747 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:13.399550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa26a7f-8b1e-40bf-87d4-95c50dc73747 admin admin] PATH_INFO: `/v3/projects/2ea87718cc274350bf823760c372804b/users/ac9d2c51bedf4b85b5c706bd6d5e30d2/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:13.408325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa26a7f-8b1e-40bf-87d4-95c50dc73747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ea87718cc274350bf823760c372804b, user_id=ac9d2c51bedf4b85b5c706bd6d5e30d2, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:13.411452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa26a7f-8b1e-40bf-87d4-95c50dc73747 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:13.449165 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 993/1957] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:13 2022] PUT /identity/v3/projects/2ea87718cc274350bf823760c372804b/users/ac9d2c51bedf4b85b5c706bd6d5e30d2/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:13.451751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5880e72-4079-46b4-825c-f17ceff382b0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:13.452760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5880e72-4079-46b4-825c-f17ceff382b0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:13.453067 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5880e72-4079-46b4-825c-f17ceff382b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:13.491566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:13.524426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f5880e72-4079-46b4-825c-f17ceff382b0 None None] MFA Rules not processed for user `ac9d2c51bedf4b85b5c706bd6d5e30d2`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:13.526037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f5880e72-4079-46b4-825c-f17ceff382b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:13.720013 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 965/1958] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:53:13.805584 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa26a7f-8b1e-40bf-87d4-95c50dc73747 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:13.807611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2aa26a7f-8b1e-40bf-87d4-95c50dc73747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:13.940072 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa26a7f-8b1e-40bf-87d4-95c50dc73747 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:13.944356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3275cc-e648-4262-ae2b-d34962fc7432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:13.945246 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3275cc-e648-4262-ae2b-d34962fc7432 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:13.945534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3275cc-e648-4262-ae2b-d34962fc7432 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:13.945857 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3275cc-e648-4262-ae2b-d34962fc7432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:13.959090 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3275cc-e648-4262-ae2b-d34962fc7432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:13.960740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3275cc-e648-4262-ae2b-d34962fc7432 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:13.984037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 994/1959] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:13 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:14.926617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5880e72-4079-46b4-825c-f17ceff382b0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:14.944065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5880e72-4079-46b4-825c-f17ceff382b0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:14.948621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db343d-038d-4156-be4e-bdb47289a1d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:14.949310 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db343d-038d-4156-be4e-bdb47289a1d7 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:14.949712 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db343d-038d-4156-be4e-bdb47289a1d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:14.950059 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db343d-038d-4156-be4e-bdb47289a1d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:15.027655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db343d-038d-4156-be4e-bdb47289a1d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:15.029139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db343d-038d-4156-be4e-bdb47289a1d7 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:15.041283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 966/1960] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:14 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:19.246102 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3275cc-e648-4262-ae2b-d34962fc7432 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:19.248021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-aa3275cc-e648-4262-ae2b-d34962fc7432 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:19.305237 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:19.341338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3275cc-e648-4262-ae2b-d34962fc7432 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:19.347713 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f619e-1211-4d67-9b9e-aeba628f61fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:19.348569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f619e-1211-4d67-9b9e-aeba628f61fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:19.348953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f619e-1211-4d67-9b9e-aeba628f61fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:19.349116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f619e-1211-4d67-9b9e-aeba628f61fb admin admin] PATH_INFO: `/v3/users/907a4d1342b748c6ad18611955d9a18e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:19.350376 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f619e-1211-4d67-9b9e-aeba628f61fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=907a4d1342b748c6ad18611955d9a18e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:19.353255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f619e-1211-4d67-9b9e-aeba628f61fb admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:19.450138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-206f619e-1211-4d67-9b9e-aeba628f61fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '907a4d1342b748c6ad18611955d9a18e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:19.466614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-206f619e-1211-4d67-9b9e-aeba628f61fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '907a4d1342b748c6ad18611955d9a18e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:19.490924 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-206f619e-1211-4d67-9b9e-aeba628f61fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '907a4d1342b748c6ad18611955d9a18e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:19.505470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-206f619e-1211-4d67-9b9e-aeba628f61fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '907a4d1342b748c6ad18611955d9a18e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:19.507945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-206f619e-1211-4d67-9b9e-aeba628f61fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '907a4d1342b748c6ad18611955d9a18e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:19.527454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 995/1961] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:19 2022] DELETE /identity/v3/users/907a4d1342b748c6ad18611955d9a18e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:19.532848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98310e2a-d1d0-4967-9f92-4bc8da5df412 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:19.533716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98310e2a-d1d0-4967-9f92-4bc8da5df412 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:19.534378 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98310e2a-d1d0-4967-9f92-4bc8da5df412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:19.589799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-98310e2a-d1d0-4967-9f92-4bc8da5df412 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:19.591112 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-98310e2a-d1d0-4967-9f92-4bc8da5df412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:19.676732 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 967/1962] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:19.694786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f619e-1211-4d67-9b9e-aeba628f61fb admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:19.696256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-206f619e-1211-4d67-9b9e-aeba628f61fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:19.781892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f619e-1211-4d67-9b9e-aeba628f61fb admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:19.786311 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48550653-9f64-4960-af33-affa0014a2b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:19.787763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48550653-9f64-4960-af33-affa0014a2b0 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:19.787763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48550653-9f64-4960-af33-affa0014a2b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:19.787763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48550653-9f64-4960-af33-affa0014a2b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:19.798409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48550653-9f64-4960-af33-affa0014a2b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:19.801083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48550653-9f64-4960-af33-affa0014a2b0 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:19.815636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 996/1963] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:20.272116 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98310e2a-d1d0-4967-9f92-4bc8da5df412 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:20.274298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-98310e2a-d1d0-4967-9f92-4bc8da5df412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:20.299238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98310e2a-d1d0-4967-9f92-4bc8da5df412 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:20.311862 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb05ca-11aa-4b03-a8fb-64212284d5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:20.312857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb05ca-11aa-4b03-a8fb-64212284d5e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:20.313167 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb05ca-11aa-4b03-a8fb-64212284d5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:20.313545 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb05ca-11aa-4b03-a8fb-64212284d5e8 admin admin] PATH_INFO: `/v3/projects/b81e9fa9f9b64d049655f13f3d523a43` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:20.314772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb05ca-11aa-4b03-a8fb-64212284d5e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b81e9fa9f9b64d049655f13f3d523a43)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:20.316958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb05ca-11aa-4b03-a8fb-64212284d5e8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:20.356079 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:20.378165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fdfb05ca-11aa-4b03-a8fb-64212284d5e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b81e9fa9f9b64d049655f13f3d523a43', 'request_id': 'req-fdfb05ca-11aa-4b03-a8fb-64212284d5e8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:20.385952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fdfb05ca-11aa-4b03-a8fb-64212284d5e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b81e9fa9f9b64d049655f13f3d523a43', 'request_id': 'req-fdfb05ca-11aa-4b03-a8fb-64212284d5e8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:20.397902 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fdfb05ca-11aa-4b03-a8fb-64212284d5e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b81e9fa9f9b64d049655f13f3d523a43', 'request_id': 'req-fdfb05ca-11aa-4b03-a8fb-64212284d5e8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:20.410376 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fdfb05ca-11aa-4b03-a8fb-64212284d5e8 admin admin] The token cache is being invalidate because project b81e9fa9f9b64d049655f13f3d523a43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:20.410690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fdfb05ca-11aa-4b03-a8fb-64212284d5e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:20.411598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fdfb05ca-11aa-4b03-a8fb-64212284d5e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:20.413520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 968/1964] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:20 2022] DELETE /identity/v3/projects/b81e9fa9f9b64d049655f13f3d523a43 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:20.418262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797777ce-3f61-4a46-bb07-9a9faf49adb9 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:20.418731 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797777ce-3f61-4a46-bb07-9a9faf49adb9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:20.418825 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797777ce-3f61-4a46-bb07-9a9faf49adb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:20.448095 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-797777ce-3f61-4a46-bb07-9a9faf49adb9 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:20.449323 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-797777ce-3f61-4a46-bb07-9a9faf49adb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:20.512634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 997/1965] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:53:20.521148 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5435c3-cf01-4e5d-b530-6a44d71487d4 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:20.521527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5435c3-cf01-4e5d-b530-6a44d71487d4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:20.521940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5435c3-cf01-4e5d-b530-6a44d71487d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:20.542244 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ac5435c3-cf01-4e5d-b530-6a44d71487d4 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:20.543455 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ac5435c3-cf01-4e5d-b530-6a44d71487d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:20.551576 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 969/1966] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:20.556901 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797777ce-3f61-4a46-bb07-9a9faf49adb9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:20.566433 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797777ce-3f61-4a46-bb07-9a9faf49adb9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:20.572262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc453a-d767-46d1-b780-f6ad0913966a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:20.573064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc453a-d767-46d1-b780-f6ad0913966a admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:20.573345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc453a-d767-46d1-b780-f6ad0913966a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:20.573618 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc453a-d767-46d1-b780-f6ad0913966a admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:20.574408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc453a-d767-46d1-b780-f6ad0913966a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:20.574714 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc453a-d767-46d1-b780-f6ad0913966a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:20.576316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc453a-d767-46d1-b780-f6ad0913966a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:20.577927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:20.590041 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 998/1967] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:53:20.594632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5435c3-cf01-4e5d-b530-6a44d71487d4 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:20.605960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5435c3-cf01-4e5d-b530-6a44d71487d4 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:20.613552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c8bbe-0f50-477d-9c18-ac59cccf6244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:20.614607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c8bbe-0f50-477d-9c18-ac59cccf6244 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:20.614991 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c8bbe-0f50-477d-9c18-ac59cccf6244 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:20.615361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c8bbe-0f50-477d-9c18-ac59cccf6244 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:20.616850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c8bbe-0f50-477d-9c18-ac59cccf6244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:20.619533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c8bbe-0f50-477d-9c18-ac59cccf6244 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:20.653589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 970/1968] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:20 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:53:20.657502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc453a-d767-46d1-b780-f6ad0913966a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:20.668521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc453a-d767-46d1-b780-f6ad0913966a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:20.747139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58517f1-9e28-451c-9aa8-fbe648606a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:20.747848 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58517f1-9e28-451c-9aa8-fbe648606a15 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:20.749480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58517f1-9e28-451c-9aa8-fbe648606a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:20.749480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58517f1-9e28-451c-9aa8-fbe648606a15 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:20.749480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58517f1-9e28-451c-9aa8-fbe648606a15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:20.751224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58517f1-9e28-451c-9aa8-fbe648606a15 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:20.780223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 999/1969] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:20 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:20.781389 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c8bbe-0f50-477d-9c18-ac59cccf6244 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:20.795877 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c8bbe-0f50-477d-9c18-ac59cccf6244 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:20.808010 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88f442-9343-45f8-a2fd-57bf24f18633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:20.808742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88f442-9343-45f8-a2fd-57bf24f18633 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:20.809137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88f442-9343-45f8-a2fd-57bf24f18633 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:20.809505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88f442-9343-45f8-a2fd-57bf24f18633 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:20.812670 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88f442-9343-45f8-a2fd-57bf24f18633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:20.814459 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88f442-9343-45f8-a2fd-57bf24f18633 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:20.828145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 971/1970] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:20.832461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58517f1-9e28-451c-9aa8-fbe648606a15 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:20.845030 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58517f1-9e28-451c-9aa8-fbe648606a15 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:20.850120 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392de701-8456-40f1-93fd-f2273cea2434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:20.851244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392de701-8456-40f1-93fd-f2273cea2434 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:20.851538 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392de701-8456-40f1-93fd-f2273cea2434 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:20.851814 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392de701-8456-40f1-93fd-f2273cea2434 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:20.852539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392de701-8456-40f1-93fd-f2273cea2434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:20.854166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392de701-8456-40f1-93fd-f2273cea2434 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:20.866488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1000/1971] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:20.870732 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88f442-9343-45f8-a2fd-57bf24f18633 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:20.881616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88f442-9343-45f8-a2fd-57bf24f18633 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:20.887875 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22120d-cb62-45cf-8f9a-89c0b2d5127c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:20.888728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22120d-cb62-45cf-8f9a-89c0b2d5127c admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:20.889403 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22120d-cb62-45cf-8f9a-89c0b2d5127c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:20.889611 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22120d-cb62-45cf-8f9a-89c0b2d5127c admin admin] PATH_INFO: `/v3/projects/2a992758f37b46a78f63590bb849df58/users/a7a2822e7af44321afe3dab9fe461412/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:20.922028 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22120d-cb62-45cf-8f9a-89c0b2d5127c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a992758f37b46a78f63590bb849df58, user_id=a7a2822e7af44321afe3dab9fe461412, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:20.925139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22120d-cb62-45cf-8f9a-89c0b2d5127c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:20.940567 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 972/1972] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:20 2022] PUT /identity/v3/projects/2a992758f37b46a78f63590bb849df58/users/a7a2822e7af44321afe3dab9fe461412/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:20.961161 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f275ef3-69aa-4528-9ecc-f1ac62b7fc7d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:20.961630 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f275ef3-69aa-4528-9ecc-f1ac62b7fc7d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:20.962070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f275ef3-69aa-4528-9ecc-f1ac62b7fc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:21.014499 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-6f275ef3-69aa-4528-9ecc-f1ac62b7fc7d None None] MFA Rules not processed for user `a7a2822e7af44321afe3dab9fe461412`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:21.016849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6f275ef3-69aa-4528-9ecc-f1ac62b7fc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:21.123637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1001/1973] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:21.128677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22120d-cb62-45cf-8f9a-89c0b2d5127c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:21.217333 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22120d-cb62-45cf-8f9a-89c0b2d5127c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:21.222838 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7618d-b9f6-4733-8d8d-473599d76df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:21.223681 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7618d-b9f6-4733-8d8d-473599d76df9 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:21.224151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7618d-b9f6-4733-8d8d-473599d76df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:21.224524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7618d-b9f6-4733-8d8d-473599d76df9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:21.225163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7618d-b9f6-4733-8d8d-473599d76df9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:21.227254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7618d-b9f6-4733-8d8d-473599d76df9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:21.258465 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 973/1974] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:21 2022] POST /identity/v3/projects => generated 375 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:53:21.263362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f275ef3-69aa-4528-9ecc-f1ac62b7fc7d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:21.282242 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f275ef3-69aa-4528-9ecc-f1ac62b7fc7d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:21.348512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2447b19-c0f1-47bf-91fa-ab413ec25fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:21.349202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2447b19-c0f1-47bf-91fa-ab413ec25fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:21.349555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2447b19-c0f1-47bf-91fa-ab413ec25fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:21.349942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2447b19-c0f1-47bf-91fa-ab413ec25fc8 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:21.352068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2447b19-c0f1-47bf-91fa-ab413ec25fc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:21.352866 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2447b19-c0f1-47bf-91fa-ab413ec25fc8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:21.362848 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:21.381122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1002/1975] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:21 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:21.386307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7618d-b9f6-4733-8d8d-473599d76df9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:21.401756 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7618d-b9f6-4733-8d8d-473599d76df9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:21.407000 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f89aec-fb04-4f54-a489-a40929fda020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:21.407704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f89aec-fb04-4f54-a489-a40929fda020 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:21.408100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f89aec-fb04-4f54-a489-a40929fda020 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:21.408445 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f89aec-fb04-4f54-a489-a40929fda020 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:21.409165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f89aec-fb04-4f54-a489-a40929fda020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:21.411142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f89aec-fb04-4f54-a489-a40929fda020 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:21.420303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2447b19-c0f1-47bf-91fa-ab413ec25fc8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:21.422480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d2447b19-c0f1-47bf-91fa-ab413ec25fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:21.427873 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 974/1976] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:53:21.432729 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f89aec-fb04-4f54-a489-a40929fda020 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:21.446223 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f89aec-fb04-4f54-a489-a40929fda020 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:21.451283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479ace4-306f-4869-ae38-eac0d77e7b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:21.452491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479ace4-306f-4869-ae38-eac0d77e7b4a admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:21.452491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479ace4-306f-4869-ae38-eac0d77e7b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:21.452772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479ace4-306f-4869-ae38-eac0d77e7b4a admin admin] PATH_INFO: `/v3/projects/937c30ed42514b169ca922cbaa739f21/users/b2bf4f1f4c5b4b80bda892141e06055b/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:21.477381 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479ace4-306f-4869-ae38-eac0d77e7b4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=937c30ed42514b169ca922cbaa739f21, user_id=b2bf4f1f4c5b4b80bda892141e06055b, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:21.480846 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479ace4-306f-4869-ae38-eac0d77e7b4a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:21.494635 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:21.502603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 975/1977] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:21 2022] PUT /identity/v3/projects/937c30ed42514b169ca922cbaa739f21/users/b2bf4f1f4c5b4b80bda892141e06055b/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:21.506756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2447b19-c0f1-47bf-91fa-ab413ec25fc8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:21.515479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479ace4-306f-4869-ae38-eac0d77e7b4a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:21.574522 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cba72-aed3-4218-b73e-fdb2eff4ffec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:21.575485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cba72-aed3-4218-b73e-fdb2eff4ffec service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:21.575770 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cba72-aed3-4218-b73e-fdb2eff4ffec service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:21.576079 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cba72-aed3-4218-b73e-fdb2eff4ffec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:21.581010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-524cba72-aed3-4218-b73e-fdb2eff4ffec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:21.615138 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479ace4-306f-4869-ae38-eac0d77e7b4a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:21.622275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f1604-af82-471b-9332-30ea09748b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:21.623225 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f1604-af82-471b-9332-30ea09748b7f admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:21.623520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f1604-af82-471b-9332-30ea09748b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:21.623909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f1604-af82-471b-9332-30ea09748b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:21.624662 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f1604-af82-471b-9332-30ea09748b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:21.626672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f1604-af82-471b-9332-30ea09748b7f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:21.640695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 976/1978] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:53:21.644649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f1604-af82-471b-9332-30ea09748b7f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:21.659961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f1604-af82-471b-9332-30ea09748b7f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:21.667767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327ced9-a291-475b-91f6-5e114f0487e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:21.668468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327ced9-a291-475b-91f6-5e114f0487e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:21.668749 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327ced9-a291-475b-91f6-5e114f0487e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:21.669174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327ced9-a291-475b-91f6-5e114f0487e6 admin admin] PATH_INFO: `/v3/projects/937c30ed42514b169ca922cbaa739f21/users/b2bf4f1f4c5b4b80bda892141e06055b/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:21.675399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327ced9-a291-475b-91f6-5e114f0487e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=937c30ed42514b169ca922cbaa739f21, user_id=b2bf4f1f4c5b4b80bda892141e06055b, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:21.678375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327ced9-a291-475b-91f6-5e114f0487e6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:21.688839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524cba72-aed3-4218-b73e-fdb2eff4ffec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:21.690352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524cba72-aed3-4218-b73e-fdb2eff4ffec service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:21.716494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 977/1979] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:21 2022] PUT /identity/v3/projects/937c30ed42514b169ca922cbaa739f21/users/b2bf4f1f4c5b4b80bda892141e06055b/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:21.719972 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1003/1980] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:53:21.728433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82e0ae-1d3c-4c09-8f64-499034b2385d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:21.728898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82e0ae-1d3c-4c09-8f64-499034b2385d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:21.729276 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82e0ae-1d3c-4c09-8f64-499034b2385d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:21.792342 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9b82e0ae-1d3c-4c09-8f64-499034b2385d None None] MFA Rules not processed for user `b2bf4f1f4c5b4b80bda892141e06055b`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:21.794853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9b82e0ae-1d3c-4c09-8f64-499034b2385d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:21.892888 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 978/1981] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:22.135666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cba72-aed3-4218-b73e-fdb2eff4ffec service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:22.137814 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-524cba72-aed3-4218-b73e-fdb2eff4ffec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:22.154930 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524cba72-aed3-4218-b73e-fdb2eff4ffec service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:22.232038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49d710-0202-4e4e-aeca-1e61dc56a3af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:22.232297 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49d710-0202-4e4e-aeca-1e61dc56a3af service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:22.232387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49d710-0202-4e4e-aeca-1e61dc56a3af service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:22.232798 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49d710-0202-4e4e-aeca-1e61dc56a3af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:22.250370 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:22.311087 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49d710-0202-4e4e-aeca-1e61dc56a3af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:22.312590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49d710-0202-4e4e-aeca-1e61dc56a3af service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:22.325122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1004/1982] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:22.683989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82e0ae-1d3c-4c09-8f64-499034b2385d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:22.686161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9b82e0ae-1d3c-4c09-8f64-499034b2385d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:22.770442 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82e0ae-1d3c-4c09-8f64-499034b2385d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:22.775890 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b7945-97f9-4835-a433-1b7110e799f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:22.776880 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b7945-97f9-4835-a433-1b7110e799f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:22.777309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b7945-97f9-4835-a433-1b7110e799f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:22.778527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819b7945-97f9-4835-a433-1b7110e799f5 admin admin] PATH_INFO: `/v3/users/bc3d3c8b185648ef8ab8528d924a5b6d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:22.779912 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b7945-97f9-4835-a433-1b7110e799f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc3d3c8b185648ef8ab8528d924a5b6d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:22.783159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819b7945-97f9-4835-a433-1b7110e799f5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:22.797604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:22.877592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-819b7945-97f9-4835-a433-1b7110e799f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc3d3c8b185648ef8ab8528d924a5b6d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:22.897268 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-819b7945-97f9-4835-a433-1b7110e799f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc3d3c8b185648ef8ab8528d924a5b6d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:22.918017 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-819b7945-97f9-4835-a433-1b7110e799f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc3d3c8b185648ef8ab8528d924a5b6d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:22.927073 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-819b7945-97f9-4835-a433-1b7110e799f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc3d3c8b185648ef8ab8528d924a5b6d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:22.928159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-819b7945-97f9-4835-a433-1b7110e799f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc3d3c8b185648ef8ab8528d924a5b6d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:22.938381 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 979/1983] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:22 2022] DELETE /identity/v3/users/bc3d3c8b185648ef8ab8528d924a5b6d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:22.942877 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eda05c-3553-40c4-8064-9f1a65a86f54 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:22.943409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eda05c-3553-40c4-8064-9f1a65a86f54 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:22.943409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eda05c-3553-40c4-8064-9f1a65a86f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:22.964767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-37eda05c-3553-40c4-8064-9f1a65a86f54 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:22.966105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-37eda05c-3553-40c4-8064-9f1a65a86f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:23.045239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1005/1984] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:23.059512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b7945-97f9-4835-a433-1b7110e799f5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:23.061945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-819b7945-97f9-4835-a433-1b7110e799f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:23.140217 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819b7945-97f9-4835-a433-1b7110e799f5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:23.144831 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd145f-1e56-4460-a59e-237db7491085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:23.145517 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd145f-1e56-4460-a59e-237db7491085 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:23.145904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd145f-1e56-4460-a59e-237db7491085 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:23.146298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd145f-1e56-4460-a59e-237db7491085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:23.159210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd145f-1e56-4460-a59e-237db7491085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:23.161196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd145f-1e56-4460-a59e-237db7491085 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:23.185867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 980/1985] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:23.617759 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eda05c-3553-40c4-8064-9f1a65a86f54 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:23.619147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-37eda05c-3553-40c4-8064-9f1a65a86f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:23.634704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eda05c-3553-40c4-8064-9f1a65a86f54 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:23.645883 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10180e1a-2059-46c3-993a-1d84937a9f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:23.647195 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10180e1a-2059-46c3-993a-1d84937a9f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:23.647619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10180e1a-2059-46c3-993a-1d84937a9f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:23.648064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10180e1a-2059-46c3-993a-1d84937a9f9d admin admin] PATH_INFO: `/v3/projects/f4d74d98fbbe4a26ad74fff76a0a7c9c` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:23.651584 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10180e1a-2059-46c3-993a-1d84937a9f9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4d74d98fbbe4a26ad74fff76a0a7c9c)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:23.654018 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10180e1a-2059-46c3-993a-1d84937a9f9d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:23.717061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-10180e1a-2059-46c3-993a-1d84937a9f9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4d74d98fbbe4a26ad74fff76a0a7c9c', 'request_id': 'req-10180e1a-2059-46c3-993a-1d84937a9f9d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:23.723257 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-10180e1a-2059-46c3-993a-1d84937a9f9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4d74d98fbbe4a26ad74fff76a0a7c9c', 'request_id': 'req-10180e1a-2059-46c3-993a-1d84937a9f9d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:23.730938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-10180e1a-2059-46c3-993a-1d84937a9f9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4d74d98fbbe4a26ad74fff76a0a7c9c', 'request_id': 'req-10180e1a-2059-46c3-993a-1d84937a9f9d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:23.754640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-10180e1a-2059-46c3-993a-1d84937a9f9d admin admin] The token cache is being invalidate because project f4d74d98fbbe4a26ad74fff76a0a7c9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:23.754932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-10180e1a-2059-46c3-993a-1d84937a9f9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:23.755584 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-10180e1a-2059-46c3-993a-1d84937a9f9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:23.757223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1006/1986] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:23 2022] DELETE /identity/v3/projects/f4d74d98fbbe4a26ad74fff76a0a7c9c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:23.761483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd145f-1e56-4460-a59e-237db7491085 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:23.772505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd145f-1e56-4460-a59e-237db7491085 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:23.894112 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e58dff-61e5-44e1-bf75-aa63678f3a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:23.895053 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e58dff-61e5-44e1-bf75-aa63678f3a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:23.895612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e58dff-61e5-44e1-bf75-aa63678f3a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:23.896136 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e58dff-61e5-44e1-bf75-aa63678f3a83 admin admin] PATH_INFO: `/v3/users/8f5c72768cd14148ae6743e82c59920c` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:23.898118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e58dff-61e5-44e1-bf75-aa63678f3a83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f5c72768cd14148ae6743e82c59920c)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:23.900173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e58dff-61e5-44e1-bf75-aa63678f3a83 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:23.950673 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:23.981349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-83e58dff-61e5-44e1-bf75-aa63678f3a83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f5c72768cd14148ae6743e82c59920c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:24.000318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-83e58dff-61e5-44e1-bf75-aa63678f3a83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f5c72768cd14148ae6743e82c59920c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:24.029209 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-83e58dff-61e5-44e1-bf75-aa63678f3a83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f5c72768cd14148ae6743e82c59920c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:24.041418 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-83e58dff-61e5-44e1-bf75-aa63678f3a83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f5c72768cd14148ae6743e82c59920c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:24.042089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-83e58dff-61e5-44e1-bf75-aa63678f3a83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f5c72768cd14148ae6743e82c59920c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:24.051932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 981/1987] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:23 2022] DELETE /identity/v3/users/8f5c72768cd14148ae6743e82c59920c => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:24.455087 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10180e1a-2059-46c3-993a-1d84937a9f9d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:24.457698 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-10180e1a-2059-46c3-993a-1d84937a9f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:24.476314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10180e1a-2059-46c3-993a-1d84937a9f9d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:24.521919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:24.546627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e2db0-aa8b-4fb4-99b2-7e7ab06a1496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:24.547374 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e2db0-aa8b-4fb4-99b2-7e7ab06a1496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:24.547725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e2db0-aa8b-4fb4-99b2-7e7ab06a1496 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:24.548100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e2db0-aa8b-4fb4-99b2-7e7ab06a1496 admin admin] PATH_INFO: `/v3/users/b657ebf3a1a2464da98daa0b621baf81` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:24.552127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e2db0-aa8b-4fb4-99b2-7e7ab06a1496 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b657ebf3a1a2464da98daa0b621baf81)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:24.554130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e2db0-aa8b-4fb4-99b2-7e7ab06a1496 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:24.612210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-217e2db0-aa8b-4fb4-99b2-7e7ab06a1496 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b657ebf3a1a2464da98daa0b621baf81'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:24.620002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-217e2db0-aa8b-4fb4-99b2-7e7ab06a1496 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b657ebf3a1a2464da98daa0b621baf81'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:24.634822 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-217e2db0-aa8b-4fb4-99b2-7e7ab06a1496 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b657ebf3a1a2464da98daa0b621baf81'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:24.644095 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-217e2db0-aa8b-4fb4-99b2-7e7ab06a1496 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b657ebf3a1a2464da98daa0b621baf81'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:24.644795 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-217e2db0-aa8b-4fb4-99b2-7e7ab06a1496 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b657ebf3a1a2464da98daa0b621baf81'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:24.652231 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1007/1988] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:24 2022] DELETE /identity/v3/users/b657ebf3a1a2464da98daa0b621baf81 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:24.656812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2225c61-b7b4-49d8-85ef-386d02d17379 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:24.657208 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2225c61-b7b4-49d8-85ef-386d02d17379 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:24.657555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2225c61-b7b4-49d8-85ef-386d02d17379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:24.690131 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b2225c61-b7b4-49d8-85ef-386d02d17379 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:24.691444 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b2225c61-b7b4-49d8-85ef-386d02d17379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:24.783099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 982/1989] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:24.799965 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e2db0-aa8b-4fb4-99b2-7e7ab06a1496 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:24.801863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-217e2db0-aa8b-4fb4-99b2-7e7ab06a1496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:24.907417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2225c61-b7b4-49d8-85ef-386d02d17379 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:24.909165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b2225c61-b7b4-49d8-85ef-386d02d17379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:24.932504 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2225c61-b7b4-49d8-85ef-386d02d17379 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:24.946563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963b980-0b4c-4dcb-84aa-f5c0e227ded9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:24.947798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963b980-0b4c-4dcb-84aa-f5c0e227ded9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:24.948255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963b980-0b4c-4dcb-84aa-f5c0e227ded9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:24.948829 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963b980-0b4c-4dcb-84aa-f5c0e227ded9 admin admin] PATH_INFO: `/v3/projects/485f53bbf73a47a4aee4b8eeb6cef862` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:24.950544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963b980-0b4c-4dcb-84aa-f5c0e227ded9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=485f53bbf73a47a4aee4b8eeb6cef862)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:24.952832 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963b980-0b4c-4dcb-84aa-f5c0e227ded9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:25.024712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e2db0-aa8b-4fb4-99b2-7e7ab06a1496 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:25.030956 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1213b-2a0c-4d3e-b795-cf86af3e20e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:25.032250 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1213b-2a0c-4d3e-b795-cf86af3e20e5 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:25.032648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1213b-2a0c-4d3e-b795-cf86af3e20e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:25.033013 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1213b-2a0c-4d3e-b795-cf86af3e20e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:25.059166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1213b-2a0c-4d3e-b795-cf86af3e20e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:25.063445 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1213b-2a0c-4d3e-b795-cf86af3e20e5 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:25.065095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e963b980-0b4c-4dcb-84aa-f5c0e227ded9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '485f53bbf73a47a4aee4b8eeb6cef862', 'request_id': 'req-e963b980-0b4c-4dcb-84aa-f5c0e227ded9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:25.077936 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e963b980-0b4c-4dcb-84aa-f5c0e227ded9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '485f53bbf73a47a4aee4b8eeb6cef862', 'request_id': 'req-e963b980-0b4c-4dcb-84aa-f5c0e227ded9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:25.105329 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e963b980-0b4c-4dcb-84aa-f5c0e227ded9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '485f53bbf73a47a4aee4b8eeb6cef862', 'request_id': 'req-e963b980-0b4c-4dcb-84aa-f5c0e227ded9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:25.110768 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1008/1990] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:25.126029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e963b980-0b4c-4dcb-84aa-f5c0e227ded9 admin admin] The token cache is being invalidate because project 485f53bbf73a47a4aee4b8eeb6cef862 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:25.126333 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e963b980-0b4c-4dcb-84aa-f5c0e227ded9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:25.128210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e963b980-0b4c-4dcb-84aa-f5c0e227ded9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:25.135564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 983/1991] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:24 2022] DELETE /identity/v3/projects/485f53bbf73a47a4aee4b8eeb6cef862 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:25.137565 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b1511-960b-42f4-b838-fd5fc469ccdd None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:25.138010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b1511-960b-42f4-b838-fd5fc469ccdd None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:25.138321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b1511-960b-42f4-b838-fd5fc469ccdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:25.176089 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-773b1511-960b-42f4-b838-fd5fc469ccdd None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:25.177415 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-773b1511-960b-42f4-b838-fd5fc469ccdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:25.291352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1009/1992] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:25.293860 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4bac9-a888-4056-9b90-0e4a8bfa63ba None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:25.294317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4bac9-a888-4056-9b90-0e4a8bfa63ba None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:25.294941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4bac9-a888-4056-9b90-0e4a8bfa63ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:25.340928 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-efb4bac9-a888-4056-9b90-0e4a8bfa63ba None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:25.345663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-efb4bac9-a888-4056-9b90-0e4a8bfa63ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:25.367635 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 984/1993] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:25.372279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b1511-960b-42f4-b838-fd5fc469ccdd None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:25.377629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:25.379317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4bac9-a888-4056-9b90-0e4a8bfa63ba None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:25.384179 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-efb4bac9-a888-4056-9b90-0e4a8bfa63ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:25.391236 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b1511-960b-42f4-b838-fd5fc469ccdd None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:25.409903 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51993fa-e460-413b-af7d-4b20d82aaca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:25.414047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51993fa-e460-413b-af7d-4b20d82aaca7 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:25.414047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51993fa-e460-413b-af7d-4b20d82aaca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:25.414047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51993fa-e460-413b-af7d-4b20d82aaca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:25.414047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51993fa-e460-413b-af7d-4b20d82aaca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:25.415489 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51993fa-e460-413b-af7d-4b20d82aaca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:25.419637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51993fa-e460-413b-af7d-4b20d82aaca7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:25.422989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4bac9-a888-4056-9b90-0e4a8bfa63ba None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:25.452921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3ce77-56c1-42d8-b996-511cf73dcf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:25.452921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3ce77-56c1-42d8-b996-511cf73dcf96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:25.452921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3ce77-56c1-42d8-b996-511cf73dcf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:25.452921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3ce77-56c1-42d8-b996-511cf73dcf96 admin admin] PATH_INFO: `/v3/users/a7a2822e7af44321afe3dab9fe461412` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:25.453621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3ce77-56c1-42d8-b996-511cf73dcf96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a2822e7af44321afe3dab9fe461412)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:25.455533 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1010/1994] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:53:25.456311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3ce77-56c1-42d8-b996-511cf73dcf96 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:25.463168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51993fa-e460-413b-af7d-4b20d82aaca7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:25.484317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51993fa-e460-413b-af7d-4b20d82aaca7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:25.496513 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acacca-336f-4ae4-868a-d9b515fac8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:25.497166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acacca-336f-4ae4-868a-d9b515fac8da admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:25.497501 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acacca-336f-4ae4-868a-d9b515fac8da admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:25.497780 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acacca-336f-4ae4-868a-d9b515fac8da admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:25.498506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3acacca-336f-4ae4-868a-d9b515fac8da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:25.501883 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3acacca-336f-4ae4-868a-d9b515fac8da admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:25.512200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:25.552290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1011/1995] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:25 2022] POST /identity/v3/projects => generated 381 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:25.554570 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acacca-336f-4ae4-868a-d9b515fac8da admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:25.555146 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-4da3ce77-56c1-42d8-b996-511cf73dcf96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a2822e7af44321afe3dab9fe461412'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:25.567783 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-4da3ce77-56c1-42d8-b996-511cf73dcf96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a2822e7af44321afe3dab9fe461412'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:25.590267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-4da3ce77-56c1-42d8-b996-511cf73dcf96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a2822e7af44321afe3dab9fe461412'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:25.603171 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-4da3ce77-56c1-42d8-b996-511cf73dcf96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a2822e7af44321afe3dab9fe461412'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:25.604050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-4da3ce77-56c1-42d8-b996-511cf73dcf96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a2822e7af44321afe3dab9fe461412'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:25.613219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 985/1996] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:25 2022] DELETE /identity/v3/users/a7a2822e7af44321afe3dab9fe461412 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:25.617868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06212d6f-4bd2-44be-8062-59e2589a7151 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:25.618411 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06212d6f-4bd2-44be-8062-59e2589a7151 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:25.618765 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06212d6f-4bd2-44be-8062-59e2589a7151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:25.650627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-06212d6f-4bd2-44be-8062-59e2589a7151 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:25.652832 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-06212d6f-4bd2-44be-8062-59e2589a7151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:25.662155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acacca-336f-4ae4-868a-d9b515fac8da admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:25.750917 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 986/1997] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:25.764464 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06212d6f-4bd2-44be-8062-59e2589a7151 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:25.766567 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-06212d6f-4bd2-44be-8062-59e2589a7151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:25.775596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95735c-d4fb-4d26-b94c-1c80c48d22fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:25.776422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95735c-d4fb-4d26-b94c-1c80c48d22fd admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:25.777497 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95735c-d4fb-4d26-b94c-1c80c48d22fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:25.778909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95735c-d4fb-4d26-b94c-1c80c48d22fd admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:25.781499 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95735c-d4fb-4d26-b94c-1c80c48d22fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:25.784430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95735c-d4fb-4d26-b94c-1c80c48d22fd admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:25.839334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1012/1998] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:25 2022] POST /identity/v3/users => generated 291 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:25.841331 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95735c-d4fb-4d26-b94c-1c80c48d22fd admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:25.850162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ed95735c-d4fb-4d26-b94c-1c80c48d22fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:25.873204 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95735c-d4fb-4d26-b94c-1c80c48d22fd admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:25.874226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06212d6f-4bd2-44be-8062-59e2589a7151 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:25.882284 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c7ced-b7c3-41c1-b667-78ba16a42e7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:25.885054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c7ced-b7c3-41c1-b667-78ba16a42e7c service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:25.885706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c7ced-b7c3-41c1-b667-78ba16a42e7c service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:25.886138 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c7ced-b7c3-41c1-b667-78ba16a42e7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:25.887769 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6547c7a-a292-4ed5-8925-ee7213954a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:25.890277 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6547c7a-a292-4ed5-8925-ee7213954a1f admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:25.890677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6547c7a-a292-4ed5-8925-ee7213954a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:25.891158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6547c7a-a292-4ed5-8925-ee7213954a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:25.893507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6547c7a-a292-4ed5-8925-ee7213954a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:25.895587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6547c7a-a292-4ed5-8925-ee7213954a1f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:25.908290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c7ced-b7c3-41c1-b667-78ba16a42e7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:25.910320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c7ced-b7c3-41c1-b667-78ba16a42e7c service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:25.940127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1013/1999] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:25.942367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 987/2000] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:25.950794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6547c7a-a292-4ed5-8925-ee7213954a1f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:25.963926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6547c7a-a292-4ed5-8925-ee7213954a1f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:25.985382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de60ce-09f4-4fcb-aef5-4fc22d552ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:25.986333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de60ce-09f4-4fcb-aef5-4fc22d552ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:25.986720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de60ce-09f4-4fcb-aef5-4fc22d552ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:25.987098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de60ce-09f4-4fcb-aef5-4fc22d552ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:25.987901 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de60ce-09f4-4fcb-aef5-4fc22d552ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:25.990499 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de60ce-09f4-4fcb-aef5-4fc22d552ce8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:26.011333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1014/2001] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:26.012666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c7ced-b7c3-41c1-b667-78ba16a42e7c service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:26.034262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c7ced-b7c3-41c1-b667-78ba16a42e7c service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:26.043859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e587e97-94c4-4800-8f32-06a49d34f777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:26.045122 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e587e97-94c4-4800-8f32-06a49d34f777 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:26.045469 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e587e97-94c4-4800-8f32-06a49d34f777 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:26.045813 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e587e97-94c4-4800-8f32-06a49d34f777 admin admin] PATH_INFO: `/v3/projects/95eb40a1778a4e17ac5285847ece631d/users/c30aa7a6fde94eeaac68efafca1d9393/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:26.079177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e587e97-94c4-4800-8f32-06a49d34f777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95eb40a1778a4e17ac5285847ece631d, user_id=c30aa7a6fde94eeaac68efafca1d9393, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:26.082073 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e587e97-94c4-4800-8f32-06a49d34f777 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:26.115950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 988/2002] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:26 2022] PUT /identity/v3/projects/95eb40a1778a4e17ac5285847ece631d/users/c30aa7a6fde94eeaac68efafca1d9393/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:26.118637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb0589-1d76-4c79-adb8-52d5a9ce70f9 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:26.118637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb0589-1d76-4c79-adb8-52d5a9ce70f9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:26.118971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb0589-1d76-4c79-adb8-52d5a9ce70f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:26.171356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-bdeb0589-1d76-4c79-adb8-52d5a9ce70f9 None None] MFA Rules not processed for user `c30aa7a6fde94eeaac68efafca1d9393`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:26.171858 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bdeb0589-1d76-4c79-adb8-52d5a9ce70f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:26.246023 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:26.325387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1015/2003] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:26.330612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e587e97-94c4-4800-8f32-06a49d34f777 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:26.366890 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:26.440643 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e587e97-94c4-4800-8f32-06a49d34f777 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:26.454326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c52c62-a166-432d-858d-5cc05c79e77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:26.454956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c52c62-a166-432d-858d-5cc05c79e77d admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:26.455487 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c52c62-a166-432d-858d-5cc05c79e77d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:26.455887 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c52c62-a166-432d-858d-5cc05c79e77d admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:26.456524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c52c62-a166-432d-858d-5cc05c79e77d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:26.459605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c52c62-a166-432d-858d-5cc05c79e77d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:26.502942 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 989/2004] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:26 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:26.506627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb0589-1d76-4c79-adb8-52d5a9ce70f9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:26.519560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb0589-1d76-4c79-adb8-52d5a9ce70f9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:26.634776 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268399c-428e-412b-94a6-0841529e1d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:26.636036 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268399c-428e-412b-94a6-0841529e1d09 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:26.636694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268399c-428e-412b-94a6-0841529e1d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:26.637239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0268399c-428e-412b-94a6-0841529e1d09 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:26.638106 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268399c-428e-412b-94a6-0841529e1d09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:26.643404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0268399c-428e-412b-94a6-0841529e1d09 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:26.677109 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1016/2005] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:26 2022] POST /identity/v3/users => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:53:26.683114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c52c62-a166-432d-858d-5cc05c79e77d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:26.702586 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c52c62-a166-432d-858d-5cc05c79e77d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:26.720352 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d620bd-720a-44ba-b818-8ac04e6b0d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:26.721048 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d620bd-720a-44ba-b818-8ac04e6b0d6f admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:26.721295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d620bd-720a-44ba-b818-8ac04e6b0d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:26.721566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d620bd-720a-44ba-b818-8ac04e6b0d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:26.722231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d620bd-720a-44ba-b818-8ac04e6b0d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:26.724012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d620bd-720a-44ba-b818-8ac04e6b0d6f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:26.739424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 990/2006] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:26.743119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268399c-428e-412b-94a6-0841529e1d09 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:26.760463 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0268399c-428e-412b-94a6-0841529e1d09 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:26.767646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae80b6c-b76c-4848-8cc7-b51591c4c22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:26.768644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae80b6c-b76c-4848-8cc7-b51591c4c22a admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:26.769033 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae80b6c-b76c-4848-8cc7-b51591c4c22a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:26.769411 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae80b6c-b76c-4848-8cc7-b51591c4c22a admin admin] PATH_INFO: `/v3/projects/e6821d623ea346e493189e0aed9f71f4/users/78694aaa9a4d47809f35ac5826f9819b/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:26.790199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae80b6c-b76c-4848-8cc7-b51591c4c22a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6821d623ea346e493189e0aed9f71f4, user_id=78694aaa9a4d47809f35ac5826f9819b, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:26.793322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae80b6c-b76c-4848-8cc7-b51591c4c22a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:26.832406 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1017/2007] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:26 2022] PUT /identity/v3/projects/e6821d623ea346e493189e0aed9f71f4/users/78694aaa9a4d47809f35ac5826f9819b/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:26.836855 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d620bd-720a-44ba-b818-8ac04e6b0d6f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:26.853633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d620bd-720a-44ba-b818-8ac04e6b0d6f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:27.003460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9e377-5895-4cf0-ae10-6d11878faa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:27.004214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9e377-5895-4cf0-ae10-6d11878faa5c admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:27.004731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9e377-5895-4cf0-ae10-6d11878faa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:27.005015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab9e377-5895-4cf0-ae10-6d11878faa5c admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:27.005712 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9e377-5895-4cf0-ae10-6d11878faa5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:27.008382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab9e377-5895-4cf0-ae10-6d11878faa5c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:27.024008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 991/2008] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:27.028250 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae80b6c-b76c-4848-8cc7-b51591c4c22a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:27.045895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9e377-5895-4cf0-ae10-6d11878faa5c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:27.048662 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae80b6c-b76c-4848-8cc7-b51591c4c22a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:27.050767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dab9e377-5895-4cf0-ae10-6d11878faa5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:27.065333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90545d-1134-4aaa-8dff-3bd9e6b7e431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:27.066249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90545d-1134-4aaa-8dff-3bd9e6b7e431 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:27.066535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90545d-1134-4aaa-8dff-3bd9e6b7e431 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:27.066812 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90545d-1134-4aaa-8dff-3bd9e6b7e431 admin admin] PATH_INFO: `/v3/projects/e6821d623ea346e493189e0aed9f71f4/users/78694aaa9a4d47809f35ac5826f9819b/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:27.072442 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90545d-1134-4aaa-8dff-3bd9e6b7e431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6821d623ea346e493189e0aed9f71f4, user_id=78694aaa9a4d47809f35ac5826f9819b, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:27.079763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90545d-1134-4aaa-8dff-3bd9e6b7e431 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:27.086389 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab9e377-5895-4cf0-ae10-6d11878faa5c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:27.131636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1018/2009] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:27 2022] PUT /identity/v3/projects/e6821d623ea346e493189e0aed9f71f4/users/78694aaa9a4d47809f35ac5826f9819b/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:27.143100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39dead-83c5-406a-b603-739c1dd796f4 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:27.144267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39dead-83c5-406a-b603-739c1dd796f4 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:27.144859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39dead-83c5-406a-b603-739c1dd796f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:27.152138 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:27.152776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:27.153060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:27.153411 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8 admin admin] PATH_INFO: `/v3/projects/3609e51cd0b14d96985bfe990965d46f` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:27.159920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3609e51cd0b14d96985bfe990965d46f)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:27.160986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:27.252923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3f39dead-83c5-406a-b603-739c1dd796f4 None None] MFA Rules not processed for user `78694aaa9a4d47809f35ac5826f9819b`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:27.254191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3f39dead-83c5-406a-b603-739c1dd796f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:27.307446 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3609e51cd0b14d96985bfe990965d46f', 'request_id': 'req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:27.315331 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3609e51cd0b14d96985bfe990965d46f', 'request_id': 'req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:27.344525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3609e51cd0b14d96985bfe990965d46f', 'request_id': 'req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:27.378990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8 admin admin] The token cache is being invalidate because project 3609e51cd0b14d96985bfe990965d46f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:27.379318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:27.380131 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:27.385493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 992/2010] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:27 2022] DELETE /identity/v3/projects/3609e51cd0b14d96985bfe990965d46f => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:27.386896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:27.388442 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:27.405419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:27.468284 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1019/2011] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:53:27.469504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39dead-83c5-406a-b603-739c1dd796f4 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:27.470810 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3f39dead-83c5-406a-b603-739c1dd796f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:27.488047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:27.542008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d6fb2-bcdd-453e-b46c-daf4caa897d8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:27.548765 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09a68d-a0a5-43bd-92e0-0c7386698130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:27.549833 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09a68d-a0a5-43bd-92e0-0c7386698130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:27.549833 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09a68d-a0a5-43bd-92e0-0c7386698130 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:27.550092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09a68d-a0a5-43bd-92e0-0c7386698130 admin admin] PATH_INFO: `/v3/projects/2a992758f37b46a78f63590bb849df58` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:27.551165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09a68d-a0a5-43bd-92e0-0c7386698130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a992758f37b46a78f63590bb849df58)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:27.553451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09a68d-a0a5-43bd-92e0-0c7386698130 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:27.582180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39dead-83c5-406a-b603-739c1dd796f4 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:27.588038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08d8ee-ff90-4f5c-98bb-445d8d0059c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:27.588825 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08d8ee-ff90-4f5c-98bb-445d8d0059c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:27.589109 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08d8ee-ff90-4f5c-98bb-445d8d0059c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:27.589383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f08d8ee-ff90-4f5c-98bb-445d8d0059c4 admin admin] PATH_INFO: `/v3/users/8df9c6322740499f8f9816f544a723d7` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:27.590791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08d8ee-ff90-4f5c-98bb-445d8d0059c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df9c6322740499f8f9816f544a723d7)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:27.593606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f08d8ee-ff90-4f5c-98bb-445d8d0059c4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:27.610309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ed09a68d-a0a5-43bd-92e0-0c7386698130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a992758f37b46a78f63590bb849df58', 'request_id': 'req-ed09a68d-a0a5-43bd-92e0-0c7386698130'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:27.618751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ed09a68d-a0a5-43bd-92e0-0c7386698130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a992758f37b46a78f63590bb849df58', 'request_id': 'req-ed09a68d-a0a5-43bd-92e0-0c7386698130'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:27.629645 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ed09a68d-a0a5-43bd-92e0-0c7386698130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a992758f37b46a78f63590bb849df58', 'request_id': 'req-ed09a68d-a0a5-43bd-92e0-0c7386698130'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:27.647245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ed09a68d-a0a5-43bd-92e0-0c7386698130 admin admin] The token cache is being invalidate because project 2a992758f37b46a78f63590bb849df58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:27.648653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ed09a68d-a0a5-43bd-92e0-0c7386698130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:27.650197 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ed09a68d-a0a5-43bd-92e0-0c7386698130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:27.655776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 993/2012] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:27 2022] DELETE /identity/v3/projects/2a992758f37b46a78f63590bb849df58 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:27.655776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09a68d-a0a5-43bd-92e0-0c7386698130 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:27.655776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ed09a68d-a0a5-43bd-92e0-0c7386698130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:27.674143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7f08d8ee-ff90-4f5c-98bb-445d8d0059c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df9c6322740499f8f9816f544a723d7'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:27.688142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7f08d8ee-ff90-4f5c-98bb-445d8d0059c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df9c6322740499f8f9816f544a723d7'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:27.711853 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7f08d8ee-ff90-4f5c-98bb-445d8d0059c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df9c6322740499f8f9816f544a723d7'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:27.722728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7f08d8ee-ff90-4f5c-98bb-445d8d0059c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df9c6322740499f8f9816f544a723d7'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:27.723503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7f08d8ee-ff90-4f5c-98bb-445d8d0059c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df9c6322740499f8f9816f544a723d7'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:27.733807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1020/2013] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:27 2022] DELETE /identity/v3/users/8df9c6322740499f8f9816f544a723d7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:27.733807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08d8ee-ff90-4f5c-98bb-445d8d0059c4 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:27.733807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7f08d8ee-ff90-4f5c-98bb-445d8d0059c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:27.736075 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09a68d-a0a5-43bd-92e0-0c7386698130 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:27.815204 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87412c06-191a-4c21-ac79-a20da0c32cf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:27.816161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87412c06-191a-4c21-ac79-a20da0c32cf5 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:27.816352 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87412c06-191a-4c21-ac79-a20da0c32cf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:27.816850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87412c06-191a-4c21-ac79-a20da0c32cf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:27.819130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-87412c06-191a-4c21-ac79-a20da0c32cf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:27.826182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f08d8ee-ff90-4f5c-98bb-445d8d0059c4 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:27.832501 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394efa1f-2c67-4128-a002-cadbae8bed1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:27.833242 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394efa1f-2c67-4128-a002-cadbae8bed1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:27.834007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394efa1f-2c67-4128-a002-cadbae8bed1b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:27.834129 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394efa1f-2c67-4128-a002-cadbae8bed1b admin admin] PATH_INFO: `/v3/users/b2bf4f1f4c5b4b80bda892141e06055b` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:27.835136 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:27.836419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394efa1f-2c67-4128-a002-cadbae8bed1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2bf4f1f4c5b4b80bda892141e06055b)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:27.841405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394efa1f-2c67-4128-a002-cadbae8bed1b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:27.935970 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:27.965983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87412c06-191a-4c21-ac79-a20da0c32cf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:27.968401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87412c06-191a-4c21-ac79-a20da0c32cf5 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:27.979780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 994/2014] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:27.995475 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-394efa1f-2c67-4128-a002-cadbae8bed1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2bf4f1f4c5b4b80bda892141e06055b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.011003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-394efa1f-2c67-4128-a002-cadbae8bed1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2bf4f1f4c5b4b80bda892141e06055b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.040294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-394efa1f-2c67-4128-a002-cadbae8bed1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2bf4f1f4c5b4b80bda892141e06055b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.056983 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-394efa1f-2c67-4128-a002-cadbae8bed1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2bf4f1f4c5b4b80bda892141e06055b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.057775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-394efa1f-2c67-4128-a002-cadbae8bed1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2bf4f1f4c5b4b80bda892141e06055b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.067256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1021/2015] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:27 2022] DELETE /identity/v3/users/b2bf4f1f4c5b4b80bda892141e06055b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:28.125737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87412c06-191a-4c21-ac79-a20da0c32cf5 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:28.127415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-87412c06-191a-4c21-ac79-a20da0c32cf5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:28.148931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87412c06-191a-4c21-ac79-a20da0c32cf5 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:28.222785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394efa1f-2c67-4128-a002-cadbae8bed1b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:28.226519 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-394efa1f-2c67-4128-a002-cadbae8bed1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:28.259341 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ce31b-2053-40c4-82b0-0e0009218358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:28.260214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ce31b-2053-40c4-82b0-0e0009218358 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:28.260717 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ce31b-2053-40c4-82b0-0e0009218358 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:28.261333 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ce31b-2053-40c4-82b0-0e0009218358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:28.263453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ad0ce31b-2053-40c4-82b0-0e0009218358 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:28.373662 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394efa1f-2c67-4128-a002-cadbae8bed1b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:28.382705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64366fe7-10ec-4986-b40b-3d7f263df948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:28.383388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64366fe7-10ec-4986-b40b-3d7f263df948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:28.383640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64366fe7-10ec-4986-b40b-3d7f263df948 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:28.383957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64366fe7-10ec-4986-b40b-3d7f263df948 admin admin] PATH_INFO: `/v3/projects/e94139dff6ec4277be7eb5960b7ec4d9` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:28.385131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64366fe7-10ec-4986-b40b-3d7f263df948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e94139dff6ec4277be7eb5960b7ec4d9)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:28.389022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64366fe7-10ec-4986-b40b-3d7f263df948 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:28.395701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ce31b-2053-40c4-82b0-0e0009218358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:28.397971 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ce31b-2053-40c4-82b0-0e0009218358 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:28.418883 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 995/2016] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:28.482727 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-64366fe7-10ec-4986-b40b-3d7f263df948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e94139dff6ec4277be7eb5960b7ec4d9', 'request_id': 'req-64366fe7-10ec-4986-b40b-3d7f263df948'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.492857 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-64366fe7-10ec-4986-b40b-3d7f263df948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e94139dff6ec4277be7eb5960b7ec4d9', 'request_id': 'req-64366fe7-10ec-4986-b40b-3d7f263df948'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.501791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-64366fe7-10ec-4986-b40b-3d7f263df948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e94139dff6ec4277be7eb5960b7ec4d9', 'request_id': 'req-64366fe7-10ec-4986-b40b-3d7f263df948'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.518469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-64366fe7-10ec-4986-b40b-3d7f263df948 admin admin] The token cache is being invalidate because project e94139dff6ec4277be7eb5960b7ec4d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:28.518820 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-64366fe7-10ec-4986-b40b-3d7f263df948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.519434 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-64366fe7-10ec-4986-b40b-3d7f263df948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.523784 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1022/2017] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:28 2022] DELETE /identity/v3/projects/e94139dff6ec4277be7eb5960b7ec4d9 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:28.525497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ce31b-2053-40c4-82b0-0e0009218358 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:28.537824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ce31b-2053-40c4-82b0-0e0009218358 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:28.560964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64366fe7-10ec-4986-b40b-3d7f263df948 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:28.563076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-64366fe7-10ec-4986-b40b-3d7f263df948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:28.612749 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677e0395-cb12-485b-958c-b127855dd2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:28.613405 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e0395-cb12-485b-958c-b127855dd2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:28.614452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e0395-cb12-485b-958c-b127855dd2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:28.614452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677e0395-cb12-485b-958c-b127855dd2e8 admin admin] PATH_INFO: `/v3/users/ac9d2c51bedf4b85b5c706bd6d5e30d2` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:28.615388 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e0395-cb12-485b-958c-b127855dd2e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac9d2c51bedf4b85b5c706bd6d5e30d2)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:28.617818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677e0395-cb12-485b-958c-b127855dd2e8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:28.620878 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:28.624840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:28.666733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64366fe7-10ec-4986-b40b-3d7f263df948 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:28.673992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0fe541-60c6-4702-b36e-e23c122fab2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:28.674692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fe541-60c6-4702-b36e-e23c122fab2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:28.674976 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fe541-60c6-4702-b36e-e23c122fab2d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:28.675286 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fe541-60c6-4702-b36e-e23c122fab2d admin admin] PATH_INFO: `/v3/projects/937c30ed42514b169ca922cbaa739f21` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:28.676484 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0fe541-60c6-4702-b36e-e23c122fab2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=937c30ed42514b169ca922cbaa739f21)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:28.678672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0fe541-60c6-4702-b36e-e23c122fab2d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:28.738367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-677e0395-cb12-485b-958c-b127855dd2e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac9d2c51bedf4b85b5c706bd6d5e30d2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.777291 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-677e0395-cb12-485b-958c-b127855dd2e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac9d2c51bedf4b85b5c706bd6d5e30d2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.820567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6b0fe541-60c6-4702-b36e-e23c122fab2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '937c30ed42514b169ca922cbaa739f21', 'request_id': 'req-6b0fe541-60c6-4702-b36e-e23c122fab2d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.832180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6b0fe541-60c6-4702-b36e-e23c122fab2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '937c30ed42514b169ca922cbaa739f21', 'request_id': 'req-6b0fe541-60c6-4702-b36e-e23c122fab2d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.842323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-677e0395-cb12-485b-958c-b127855dd2e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac9d2c51bedf4b85b5c706bd6d5e30d2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.854006 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-677e0395-cb12-485b-958c-b127855dd2e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac9d2c51bedf4b85b5c706bd6d5e30d2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.855503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-677e0395-cb12-485b-958c-b127855dd2e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac9d2c51bedf4b85b5c706bd6d5e30d2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.857248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6b0fe541-60c6-4702-b36e-e23c122fab2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '937c30ed42514b169ca922cbaa739f21', 'request_id': 'req-6b0fe541-60c6-4702-b36e-e23c122fab2d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.867166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 996/2018] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:28 2022] DELETE /identity/v3/users/ac9d2c51bedf4b85b5c706bd6d5e30d2 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:28.908782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6b0fe541-60c6-4702-b36e-e23c122fab2d admin admin] The token cache is being invalidate because project 937c30ed42514b169ca922cbaa739f21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:28.909082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6b0fe541-60c6-4702-b36e-e23c122fab2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.916470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6b0fe541-60c6-4702-b36e-e23c122fab2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:28.928480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1023/2019] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:28 2022] DELETE /identity/v3/projects/937c30ed42514b169ca922cbaa739f21 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:28.930386 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce913a-582b-47cd-81b5-bb7513d7fb72 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:28.930684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce913a-582b-47cd-81b5-bb7513d7fb72 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:28.933560 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce913a-582b-47cd-81b5-bb7513d7fb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:28.999657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f1ce913a-582b-47cd-81b5-bb7513d7fb72 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:29.001218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f1ce913a-582b-47cd-81b5-bb7513d7fb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:29.129268 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 997/2020] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:29.129891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025ad9d-bedf-442d-869d-f631ccbbef2c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:29.129891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025ad9d-bedf-442d-869d-f631ccbbef2c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:29.130029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025ad9d-bedf-442d-869d-f631ccbbef2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:29.153055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a025ad9d-bedf-442d-869d-f631ccbbef2c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:29.154315 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a025ad9d-bedf-442d-869d-f631ccbbef2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:29.167866 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1024/2021] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:29.169752 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce913a-582b-47cd-81b5-bb7513d7fb72 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:29.180168 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce913a-582b-47cd-81b5-bb7513d7fb72 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:29.187283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb5483-b44b-4183-961f-5ed2a10efba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:29.188001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb5483-b44b-4183-961f-5ed2a10efba4 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:29.188298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb5483-b44b-4183-961f-5ed2a10efba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:29.188572 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb5483-b44b-4183-961f-5ed2a10efba4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:29.189269 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb5483-b44b-4183-961f-5ed2a10efba4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:29.189573 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb5483-b44b-4183-961f-5ed2a10efba4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:29.191508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb5483-b44b-4183-961f-5ed2a10efba4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:29.208582 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 998/2022] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:53:29.211923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025ad9d-bedf-442d-869d-f631ccbbef2c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:29.221569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025ad9d-bedf-442d-869d-f631ccbbef2c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:29.230710 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddca04-ea6f-44b7-a48d-86a181400951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:29.231408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddca04-ea6f-44b7-a48d-86a181400951 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:29.231637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddca04-ea6f-44b7-a48d-86a181400951 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:29.231944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddca04-ea6f-44b7-a48d-86a181400951 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:29.232594 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddca04-ea6f-44b7-a48d-86a181400951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:29.234780 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddca04-ea6f-44b7-a48d-86a181400951 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:29.275392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1025/2023] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:29 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:29.279902 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb5483-b44b-4183-961f-5ed2a10efba4 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:29.294532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb5483-b44b-4183-961f-5ed2a10efba4 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:29.365626 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b8b3e-358b-4adc-8bbc-ee1d3f421879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:29.366807 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b8b3e-358b-4adc-8bbc-ee1d3f421879 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:29.367215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b8b3e-358b-4adc-8bbc-ee1d3f421879 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:29.367493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48b8b3e-358b-4adc-8bbc-ee1d3f421879 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:29.368170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b8b3e-358b-4adc-8bbc-ee1d3f421879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:29.370759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48b8b3e-358b-4adc-8bbc-ee1d3f421879 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:29.392209 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:29.405008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddca04-ea6f-44b7-a48d-86a181400951 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:29.406025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 999/2024] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:29 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:29.418741 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddca04-ea6f-44b7-a48d-86a181400951 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:29.430248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0e253-effd-4a84-aa96-f6cc2fca22ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:29.430897 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0e253-effd-4a84-aa96-f6cc2fca22ea admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:29.431477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0e253-effd-4a84-aa96-f6cc2fca22ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:29.431767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe0e253-effd-4a84-aa96-f6cc2fca22ea admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:29.436697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0e253-effd-4a84-aa96-f6cc2fca22ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:29.438819 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe0e253-effd-4a84-aa96-f6cc2fca22ea admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:29.455477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1026/2025] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:53:29.460474 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b8b3e-358b-4adc-8bbc-ee1d3f421879 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:29.481797 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48b8b3e-358b-4adc-8bbc-ee1d3f421879 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:29.488425 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505555dd-a0fc-4aa1-af84-daeed339208f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:29.489063 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505555dd-a0fc-4aa1-af84-daeed339208f admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:29.489343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505555dd-a0fc-4aa1-af84-daeed339208f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:29.489767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505555dd-a0fc-4aa1-af84-daeed339208f admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:29.490604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505555dd-a0fc-4aa1-af84-daeed339208f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:29.492784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505555dd-a0fc-4aa1-af84-daeed339208f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:29.519870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1000/2026] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:29.530884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0e253-effd-4a84-aa96-f6cc2fca22ea admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:29.541308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe0e253-effd-4a84-aa96-f6cc2fca22ea admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:29.555153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf25bb-7278-45f7-bc1b-4f99a7b819aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:29.555969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf25bb-7278-45f7-bc1b-4f99a7b819aa admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:29.556519 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf25bb-7278-45f7-bc1b-4f99a7b819aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:29.557213 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf25bb-7278-45f7-bc1b-4f99a7b819aa admin admin] PATH_INFO: `/v3/projects/33e2ac6867e748d4af2042a95b5d0b58/users/88d1c9c7ec5e409698fdf7bad80210c5/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:29.574906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf25bb-7278-45f7-bc1b-4f99a7b819aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e2ac6867e748d4af2042a95b5d0b58, user_id=88d1c9c7ec5e409698fdf7bad80210c5, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:29.577645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf25bb-7278-45f7-bc1b-4f99a7b819aa admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:29.582690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505555dd-a0fc-4aa1-af84-daeed339208f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:29.584528 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-505555dd-a0fc-4aa1-af84-daeed339208f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:29.600193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505555dd-a0fc-4aa1-af84-daeed339208f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:29.608860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1027/2027] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:29 2022] PUT /identity/v3/projects/33e2ac6867e748d4af2042a95b5d0b58/users/88d1c9c7ec5e409698fdf7bad80210c5/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:29.625913 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97e70c-347f-43a5-bf92-9466ece4a405 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:29.626563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97e70c-347f-43a5-bf92-9466ece4a405 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:29.626563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97e70c-347f-43a5-bf92-9466ece4a405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:29.669371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4b97e70c-347f-43a5-bf92-9466ece4a405 None None] MFA Rules not processed for user `88d1c9c7ec5e409698fdf7bad80210c5`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:29.670603 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4b97e70c-347f-43a5-bf92-9466ece4a405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:29.701649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209202b-1d04-412e-93f5-9320afd91b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:29.702798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209202b-1d04-412e-93f5-9320afd91b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:29.703161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209202b-1d04-412e-93f5-9320afd91b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:29.703509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209202b-1d04-412e-93f5-9320afd91b89 admin admin] PATH_INFO: `/v3/projects/2ea87718cc274350bf823760c372804b` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:29.704822 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209202b-1d04-412e-93f5-9320afd91b89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ea87718cc274350bf823760c372804b)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:29.706991 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209202b-1d04-412e-93f5-9320afd91b89 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:29.757766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:29.771678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3209202b-1d04-412e-93f5-9320afd91b89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ea87718cc274350bf823760c372804b', 'request_id': 'req-3209202b-1d04-412e-93f5-9320afd91b89'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:29.778440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1028/2028] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:29.786207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3209202b-1d04-412e-93f5-9320afd91b89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ea87718cc274350bf823760c372804b', 'request_id': 'req-3209202b-1d04-412e-93f5-9320afd91b89'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:29.789717 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97e70c-347f-43a5-bf92-9466ece4a405 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:29.793705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4b97e70c-347f-43a5-bf92-9466ece4a405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:29.798485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3209202b-1d04-412e-93f5-9320afd91b89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ea87718cc274350bf823760c372804b', 'request_id': 'req-3209202b-1d04-412e-93f5-9320afd91b89'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:29.814361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3209202b-1d04-412e-93f5-9320afd91b89 admin admin] The token cache is being invalidate because project 2ea87718cc274350bf823760c372804b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:29.814984 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3209202b-1d04-412e-93f5-9320afd91b89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:29.815681 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3209202b-1d04-412e-93f5-9320afd91b89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:29.817537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1001/2029] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:29 2022] DELETE /identity/v3/projects/2ea87718cc274350bf823760c372804b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:29.823942 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2b8bb-451e-48b4-820e-ee295459e316 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:29.824693 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2b8bb-451e-48b4-820e-ee295459e316 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:29.824989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2b8bb-451e-48b4-820e-ee295459e316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:29.851643 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c5f2b8bb-451e-48b4-820e-ee295459e316 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:29.853360 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c5f2b8bb-451e-48b4-820e-ee295459e316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:29.869588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97e70c-347f-43a5-bf92-9466ece4a405 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:29.932844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1002/2030] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:53:29.940664 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279b8ab-b856-4dc5-937f-d890553d7e5b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:29.941129 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279b8ab-b856-4dc5-937f-d890553d7e5b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:29.941553 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279b8ab-b856-4dc5-937f-d890553d7e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:29.945765 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c91e8-fe80-46d9-9003-62f6c0600a26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:29.946321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c91e8-fe80-46d9-9003-62f6c0600a26 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:29.946606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c91e8-fe80-46d9-9003-62f6c0600a26 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:29.946876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2c91e8-fe80-46d9-9003-62f6c0600a26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:29.948863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0d2c91e8-fe80-46d9-9003-62f6c0600a26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:29.966196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1279b8ab-b856-4dc5-937f-d890553d7e5b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:29.967456 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1279b8ab-b856-4dc5-937f-d890553d7e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:29.983963 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1003/2031] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:53:29.985882 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279b8ab-b856-4dc5-937f-d890553d7e5b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:29.996141 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1279b8ab-b856-4dc5-937f-d890553d7e5b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:30.001070 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476adc1-e30d-47f2-8e7a-739dd297afe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:30.001829 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476adc1-e30d-47f2-8e7a-739dd297afe8 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:30.002152 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476adc1-e30d-47f2-8e7a-739dd297afe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:30.002478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476adc1-e30d-47f2-8e7a-739dd297afe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:30.003216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476adc1-e30d-47f2-8e7a-739dd297afe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:30.003426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476adc1-e30d-47f2-8e7a-739dd297afe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:30.005351 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476adc1-e30d-47f2-8e7a-739dd297afe8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:30.019751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:30.028018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1004/2032] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:53:30.032305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476adc1-e30d-47f2-8e7a-739dd297afe8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:30.037044 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c91e8-fe80-46d9-9003-62f6c0600a26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:30.039098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2c91e8-fe80-46d9-9003-62f6c0600a26 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:30.046418 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476adc1-e30d-47f2-8e7a-739dd297afe8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:30.052095 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1029/2033] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:30.052439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a31a0-8a6d-4baf-bc4e-1383beebda41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:30.053450 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a31a0-8a6d-4baf-bc4e-1383beebda41 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:30.054493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a31a0-8a6d-4baf-bc4e-1383beebda41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:30.054844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a31a0-8a6d-4baf-bc4e-1383beebda41 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:30.055557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a31a0-8a6d-4baf-bc4e-1383beebda41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:30.058210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a31a0-8a6d-4baf-bc4e-1383beebda41 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:30.104024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1005/2034] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:30 2022] POST /identity/v3/projects => generated 387 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:30.104485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c91e8-fe80-46d9-9003-62f6c0600a26 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:30.132479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2c91e8-fe80-46d9-9003-62f6c0600a26 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:30.228521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1937d-2a6a-43a1-95e5-3a8bf6a6b500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:30.230117 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1937d-2a6a-43a1-95e5-3a8bf6a6b500 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:30.230631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1937d-2a6a-43a1-95e5-3a8bf6a6b500 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:30.231211 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1937d-2a6a-43a1-95e5-3a8bf6a6b500 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:30.232475 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1937d-2a6a-43a1-95e5-3a8bf6a6b500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:30.234938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1937d-2a6a-43a1-95e5-3a8bf6a6b500 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:30.246870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:30.267488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1030/2035] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:30 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:30.268290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a31a0-8a6d-4baf-bc4e-1383beebda41 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:30.285888 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a31a0-8a6d-4baf-bc4e-1383beebda41 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:30.301884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e8f6b-d69a-420e-baf1-a3cc5d8b78fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:30.302529 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e8f6b-d69a-420e-baf1-a3cc5d8b78fc admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:30.302871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e8f6b-d69a-420e-baf1-a3cc5d8b78fc admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:30.303224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e8f6b-d69a-420e-baf1-a3cc5d8b78fc admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:30.304221 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e8f6b-d69a-420e-baf1-a3cc5d8b78fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:30.306938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e8f6b-d69a-420e-baf1-a3cc5d8b78fc admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:30.332011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1937d-2a6a-43a1-95e5-3a8bf6a6b500 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:30.332798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1006/2036] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:53:30.344803 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1937d-2a6a-43a1-95e5-3a8bf6a6b500 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:30.350291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e10d5fb-d87e-4b7d-8ee1-257af7249cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:30.351813 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10d5fb-d87e-4b7d-8ee1-257af7249cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:30.352532 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10d5fb-d87e-4b7d-8ee1-257af7249cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:30.352875 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e10d5fb-d87e-4b7d-8ee1-257af7249cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:30.353653 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10d5fb-d87e-4b7d-8ee1-257af7249cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:30.356084 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e10d5fb-d87e-4b7d-8ee1-257af7249cd4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:30.369239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1031/2037] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:30.373523 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e8f6b-d69a-420e-baf1-a3cc5d8b78fc admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:30.385636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e8f6b-d69a-420e-baf1-a3cc5d8b78fc admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:30.392954 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5bfee-5252-430e-9466-e501e959a43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:30.393637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5bfee-5252-430e-9466-e501e959a43b admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:30.393920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5bfee-5252-430e-9466-e501e959a43b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:30.394269 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5bfee-5252-430e-9466-e501e959a43b admin admin] PATH_INFO: `/v3/projects/7dff918de1654017bf21ba8ef2b891c5/users/10599506f7a74caaba8e067b122ba8a6/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:30.420964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b5bfee-5252-430e-9466-e501e959a43b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dff918de1654017bf21ba8ef2b891c5, user_id=10599506f7a74caaba8e067b122ba8a6, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:30.436632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b5bfee-5252-430e-9466-e501e959a43b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:30.465875 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1007/2038] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:30 2022] PUT /identity/v3/projects/7dff918de1654017bf21ba8ef2b891c5/users/10599506f7a74caaba8e067b122ba8a6/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:30.478931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380741bb-11f7-4175-9b8a-5d05e3f5b1df None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:30.479055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380741bb-11f7-4175-9b8a-5d05e3f5b1df None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:30.479285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380741bb-11f7-4175-9b8a-5d05e3f5b1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:30.533006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-380741bb-11f7-4175-9b8a-5d05e3f5b1df None None] MFA Rules not processed for user `10599506f7a74caaba8e067b122ba8a6`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:30.536210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-380741bb-11f7-4175-9b8a-5d05e3f5b1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:30.631099 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1032/2039] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:30.639761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5bfee-5252-430e-9466-e501e959a43b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:30.700028 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380741bb-11f7-4175-9b8a-5d05e3f5b1df None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:30.702480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-380741bb-11f7-4175-9b8a-5d05e3f5b1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:30.732511 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5bfee-5252-430e-9466-e501e959a43b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:30.737540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5dc0e-e37c-4897-a70f-a36fea880296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:30.738216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5dc0e-e37c-4897-a70f-a36fea880296 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:30.738510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5dc0e-e37c-4897-a70f-a36fea880296 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:30.738792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5dc0e-e37c-4897-a70f-a36fea880296 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:30.739460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d5dc0e-e37c-4897-a70f-a36fea880296 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:30.741447 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d5dc0e-e37c-4897-a70f-a36fea880296 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:30.771606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1008/2040] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:30 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:30.776024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5dc0e-e37c-4897-a70f-a36fea880296 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:30.788985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380741bb-11f7-4175-9b8a-5d05e3f5b1df None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:30.842378 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5dc0e-e37c-4897-a70f-a36fea880296 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:30.847045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fd74e-6cd8-4090-9002-426d006f23d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:30.847663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fd74e-6cd8-4090-9002-426d006f23d7 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:30.847963 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fd74e-6cd8-4090-9002-426d006f23d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:30.848237 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fd74e-6cd8-4090-9002-426d006f23d7 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:30.848897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fd74e-6cd8-4090-9002-426d006f23d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:30.850282 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e76caa-4be8-4389-9074-d48f19a265eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:30.850813 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fd74e-6cd8-4090-9002-426d006f23d7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:30.851081 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e76caa-4be8-4389-9074-d48f19a265eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:30.851549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e76caa-4be8-4389-9074-d48f19a265eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:30.851928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e76caa-4be8-4389-9074-d48f19a265eb admin admin] PATH_INFO: `/v3/users/5839cda8aa894d0093782221b42dbb50` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:30.853223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e76caa-4be8-4389-9074-d48f19a265eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5839cda8aa894d0093782221b42dbb50)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:30.855310 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e76caa-4be8-4389-9074-d48f19a265eb admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:30.875684 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1009/2041] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:30 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:53:30.879620 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fd74e-6cd8-4090-9002-426d006f23d7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:30.880979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:INSERT=1,keystone:SELECT=96,keystone:DELETE=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:30.890800 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fd74e-6cd8-4090-9002-426d006f23d7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:30.896596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40316e6-dcd1-4a94-8135-c0968cb7d850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:30.897240 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40316e6-dcd1-4a94-8135-c0968cb7d850 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:30.897550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40316e6-dcd1-4a94-8135-c0968cb7d850 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:30.897869 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40316e6-dcd1-4a94-8135-c0968cb7d850 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:30.898504 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40316e6-dcd1-4a94-8135-c0968cb7d850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:30.900145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40316e6-dcd1-4a94-8135-c0968cb7d850 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:30.911001 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b6e76caa-4be8-4389-9074-d48f19a265eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5839cda8aa894d0093782221b42dbb50'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:30.911873 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1010/2042] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:53:30.916338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40316e6-dcd1-4a94-8135-c0968cb7d850 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:30.921866 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b6e76caa-4be8-4389-9074-d48f19a265eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5839cda8aa894d0093782221b42dbb50'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:30.922583 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:30.926539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40316e6-dcd1-4a94-8135-c0968cb7d850 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:30.933036 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15cf34-b3f8-477b-ba9f-6de9694f5280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:30.933761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15cf34-b3f8-477b-ba9f-6de9694f5280 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:30.934113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15cf34-b3f8-477b-ba9f-6de9694f5280 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:30.934452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15cf34-b3f8-477b-ba9f-6de9694f5280 admin admin] PATH_INFO: `/v3/projects/1fdb9363081a4daba6c8d537b38ba589/users/8b0755a8bf534c8e97543b65bed3ac3e/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:30.935505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b6e76caa-4be8-4389-9074-d48f19a265eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5839cda8aa894d0093782221b42dbb50'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:30.943322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b6e76caa-4be8-4389-9074-d48f19a265eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5839cda8aa894d0093782221b42dbb50'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:30.944008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b6e76caa-4be8-4389-9074-d48f19a265eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5839cda8aa894d0093782221b42dbb50'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:30.950808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1033/2043] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:30 2022] DELETE /identity/v3/users/5839cda8aa894d0093782221b42dbb50 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:30.950934 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15cf34-b3f8-477b-ba9f-6de9694f5280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fdb9363081a4daba6c8d537b38ba589, user_id=8b0755a8bf534c8e97543b65bed3ac3e, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:30.953823 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15cf34-b3f8-477b-ba9f-6de9694f5280 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:30.968022 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1011/2044] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:30 2022] PUT /identity/v3/projects/1fdb9363081a4daba6c8d537b38ba589/users/8b0755a8bf534c8e97543b65bed3ac3e/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:30.972026 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e76caa-4be8-4389-9074-d48f19a265eb admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:30.973409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b6e76caa-4be8-4389-9074-d48f19a265eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:31.055730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e76caa-4be8-4389-9074-d48f19a265eb admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:31.118200 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115f402-2b1b-4faf-a184-f573bb07974a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:31.118809 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115f402-2b1b-4faf-a184-f573bb07974a admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:31.119086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115f402-2b1b-4faf-a184-f573bb07974a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:31.119356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115f402-2b1b-4faf-a184-f573bb07974a admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:31.120017 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5115f402-2b1b-4faf-a184-f573bb07974a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:31.121625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5115f402-2b1b-4faf-a184-f573bb07974a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:31.133586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1034/2045] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:30 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:31.137659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15cf34-b3f8-477b-ba9f-6de9694f5280 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:31.143934 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115f402-2b1b-4faf-a184-f573bb07974a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:31.146173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5115f402-2b1b-4faf-a184-f573bb07974a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:31.156624 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15cf34-b3f8-477b-ba9f-6de9694f5280 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:31.161804 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5115f402-2b1b-4faf-a184-f573bb07974a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:31.167980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b1ab0-a145-4a78-9174-804f860ebaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:31.168631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b1ab0-a145-4a78-9174-804f860ebaa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:31.168911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b1ab0-a145-4a78-9174-804f860ebaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:31.169187 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b1ab0-a145-4a78-9174-804f860ebaa8 admin admin] PATH_INFO: `/v3/projects/c50ff4383b5c43788acb95c7c0a3d56f` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:31.170324 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b1ab0-a145-4a78-9174-804f860ebaa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c50ff4383b5c43788acb95c7c0a3d56f)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:31.172291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b1ab0-a145-4a78-9174-804f860ebaa8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:31.172481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c1830-bc77-4efa-94da-96bf4199a537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:31.173162 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c1830-bc77-4efa-94da-96bf4199a537 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:31.173442 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c1830-bc77-4efa-94da-96bf4199a537 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:31.173727 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c1830-bc77-4efa-94da-96bf4199a537 admin admin] PATH_INFO: `/v3/projects/1fdb9363081a4daba6c8d537b38ba589/users/8b0755a8bf534c8e97543b65bed3ac3e/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:31.176291 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c1830-bc77-4efa-94da-96bf4199a537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fdb9363081a4daba6c8d537b38ba589, user_id=8b0755a8bf534c8e97543b65bed3ac3e, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:31.179920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c1830-bc77-4efa-94da-96bf4199a537 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:31.190633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1012/2046] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:31 2022] PUT /identity/v3/projects/1fdb9363081a4daba6c8d537b38ba589/users/8b0755a8bf534c8e97543b65bed3ac3e/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:31.200771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa246a0d-57a1-4c63-a6e8-0fdd0f227162 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:31.201122 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa246a0d-57a1-4c63-a6e8-0fdd0f227162 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:31.201464 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa246a0d-57a1-4c63-a6e8-0fdd0f227162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:31.214019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:31.226679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-966b1ab0-a145-4a78-9174-804f860ebaa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c50ff4383b5c43788acb95c7c0a3d56f', 'request_id': 'req-966b1ab0-a145-4a78-9174-804f860ebaa8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.232023 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-aa246a0d-57a1-4c63-a6e8-0fdd0f227162 None None] MFA Rules not processed for user `8b0755a8bf534c8e97543b65bed3ac3e`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:31.233259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-aa246a0d-57a1-4c63-a6e8-0fdd0f227162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:31.233459 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-966b1ab0-a145-4a78-9174-804f860ebaa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c50ff4383b5c43788acb95c7c0a3d56f', 'request_id': 'req-966b1ab0-a145-4a78-9174-804f860ebaa8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.241153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-966b1ab0-a145-4a78-9174-804f860ebaa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c50ff4383b5c43788acb95c7c0a3d56f', 'request_id': 'req-966b1ab0-a145-4a78-9174-804f860ebaa8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.255518 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-966b1ab0-a145-4a78-9174-804f860ebaa8 admin admin] The token cache is being invalidate because project c50ff4383b5c43788acb95c7c0a3d56f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:31.255807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-966b1ab0-a145-4a78-9174-804f860ebaa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.256514 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-966b1ab0-a145-4a78-9174-804f860ebaa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.258068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1035/2047] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:31 2022] DELETE /identity/v3/projects/c50ff4383b5c43788acb95c7c0a3d56f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:31.261915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b1ab0-a145-4a78-9174-804f860ebaa8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:31.263256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-966b1ab0-a145-4a78-9174-804f860ebaa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:31.307517 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1013/2048] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:53:31.318331 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa246a0d-57a1-4c63-a6e8-0fdd0f227162 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:31.319748 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-aa246a0d-57a1-4c63-a6e8-0fdd0f227162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:31.330364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b1ab0-a145-4a78-9174-804f860ebaa8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:31.335327 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d3eb7-1b1d-42b6-8227-b7f9b2c73f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:31.335927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d3eb7-1b1d-42b6-8227-b7f9b2c73f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:31.336132 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d3eb7-1b1d-42b6-8227-b7f9b2c73f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:31.336402 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d3eb7-1b1d-42b6-8227-b7f9b2c73f10 admin admin] PATH_INFO: `/v3/users/06c9a34f6052478f9ad58e90b3b53f42` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:31.337527 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d3eb7-1b1d-42b6-8227-b7f9b2c73f10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06c9a34f6052478f9ad58e90b3b53f42)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:31.339545 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d3eb7-1b1d-42b6-8227-b7f9b2c73f10 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:31.390535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa246a0d-57a1-4c63-a6e8-0fdd0f227162 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:31.394303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1f4d3eb7-1b1d-42b6-8227-b7f9b2c73f10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06c9a34f6052478f9ad58e90b3b53f42'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.394561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eee4e9-306d-4ce8-90bd-b9c50baeeef9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:31.395174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eee4e9-306d-4ce8-90bd-b9c50baeeef9 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:31.395455 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eee4e9-306d-4ce8-90bd-b9c50baeeef9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:31.395721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eee4e9-306d-4ce8-90bd-b9c50baeeef9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:31.401773 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1f4d3eb7-1b1d-42b6-8227-b7f9b2c73f10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06c9a34f6052478f9ad58e90b3b53f42'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.406078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eee4e9-306d-4ce8-90bd-b9c50baeeef9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:31.407517 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eee4e9-306d-4ce8-90bd-b9c50baeeef9 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:31.414748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1f4d3eb7-1b1d-42b6-8227-b7f9b2c73f10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06c9a34f6052478f9ad58e90b3b53f42'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.418144 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1014/2049] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:53:31.420471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eee4e9-306d-4ce8-90bd-b9c50baeeef9 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:31.421780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-44eee4e9-306d-4ce8-90bd-b9c50baeeef9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:31.423779 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1f4d3eb7-1b1d-42b6-8227-b7f9b2c73f10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06c9a34f6052478f9ad58e90b3b53f42'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.424622 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1f4d3eb7-1b1d-42b6-8227-b7f9b2c73f10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06c9a34f6052478f9ad58e90b3b53f42'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.432695 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1036/2050] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:31 2022] DELETE /identity/v3/users/06c9a34f6052478f9ad58e90b3b53f42 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:31.438152 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eee4e9-306d-4ce8-90bd-b9c50baeeef9 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:31.500167 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:31.514601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2e6c9-8c8d-436f-bb1b-a57007105da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:31.515257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2e6c9-8c8d-436f-bb1b-a57007105da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:31.515756 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2e6c9-8c8d-436f-bb1b-a57007105da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:31.516204 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2e6c9-8c8d-436f-bb1b-a57007105da4 admin admin] PATH_INFO: `/v3/users/c30aa7a6fde94eeaac68efafca1d9393` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:31.517552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2e6c9-8c8d-436f-bb1b-a57007105da4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c30aa7a6fde94eeaac68efafca1d9393)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:31.519575 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2e6c9-8c8d-436f-bb1b-a57007105da4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:31.598607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5dc2e6c9-8c8d-436f-bb1b-a57007105da4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c30aa7a6fde94eeaac68efafca1d9393'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.619983 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d3eb7-1b1d-42b6-8227-b7f9b2c73f10 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:31.622522 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1f4d3eb7-1b1d-42b6-8227-b7f9b2c73f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:31.628160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5dc2e6c9-8c8d-436f-bb1b-a57007105da4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c30aa7a6fde94eeaac68efafca1d9393'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.638978 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d3eb7-1b1d-42b6-8227-b7f9b2c73f10 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:31.651417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb2d96-653f-4653-a279-6a452aa02cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:31.652307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5dc2e6c9-8c8d-436f-bb1b-a57007105da4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c30aa7a6fde94eeaac68efafca1d9393'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.653379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb2d96-653f-4653-a279-6a452aa02cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:31.655506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb2d96-653f-4653-a279-6a452aa02cea admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:31.655927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cb2d96-653f-4653-a279-6a452aa02cea admin admin] PATH_INFO: `/v3/users/10599506f7a74caaba8e067b122ba8a6` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:31.659186 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb2d96-653f-4653-a279-6a452aa02cea admin admin] RBAC: Authorizing `identity:delete_user(user_id=10599506f7a74caaba8e067b122ba8a6)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:31.664907 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cb2d96-653f-4653-a279-6a452aa02cea admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:31.672766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5dc2e6c9-8c8d-436f-bb1b-a57007105da4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c30aa7a6fde94eeaac68efafca1d9393'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.673559 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5dc2e6c9-8c8d-436f-bb1b-a57007105da4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c30aa7a6fde94eeaac68efafca1d9393'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.685400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1015/2051] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:31 2022] DELETE /identity/v3/users/c30aa7a6fde94eeaac68efafca1d9393 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:31.692836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467571c4-1060-412e-946b-bd5973c039f1 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:31.693139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467571c4-1060-412e-946b-bd5973c039f1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:31.696131 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467571c4-1060-412e-946b-bd5973c039f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:31.753555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-467571c4-1060-412e-946b-bd5973c039f1 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:31.755142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-467571c4-1060-412e-946b-bd5973c039f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:31.755262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:31.771446 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-22cb2d96-653f-4653-a279-6a452aa02cea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10599506f7a74caaba8e067b122ba8a6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.783253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-22cb2d96-653f-4653-a279-6a452aa02cea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10599506f7a74caaba8e067b122ba8a6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.820849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-22cb2d96-653f-4653-a279-6a452aa02cea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10599506f7a74caaba8e067b122ba8a6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.838756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-22cb2d96-653f-4653-a279-6a452aa02cea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10599506f7a74caaba8e067b122ba8a6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.839645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-22cb2d96-653f-4653-a279-6a452aa02cea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10599506f7a74caaba8e067b122ba8a6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:31.847272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1037/2052] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:31 2022] DELETE /identity/v3/users/10599506f7a74caaba8e067b122ba8a6 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:31.852562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb2d96-653f-4653-a279-6a452aa02cea admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:31.854350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-22cb2d96-653f-4653-a279-6a452aa02cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:31.869432 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1016/2053] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:31.873226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87722cf4-0a8a-4b30-82ac-ae9fdc3ba4ee None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:31.873226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87722cf4-0a8a-4b30-82ac-ae9fdc3ba4ee None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:31.873226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87722cf4-0a8a-4b30-82ac-ae9fdc3ba4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:31.894576 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-87722cf4-0a8a-4b30-82ac-ae9fdc3ba4ee None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:31.896281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-87722cf4-0a8a-4b30-82ac-ae9fdc3ba4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:31.950965 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cb2d96-653f-4653-a279-6a452aa02cea admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:31.955893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868edd6a-ec12-4735-a266-255c6047435f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:31.956545 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868edd6a-ec12-4735-a266-255c6047435f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:31.956831 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868edd6a-ec12-4735-a266-255c6047435f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:31.957113 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868edd6a-ec12-4735-a266-255c6047435f admin admin] PATH_INFO: `/v3/projects/63a2ceed7faa45fba9c6fbf4f03c2d9e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:31.958253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868edd6a-ec12-4735-a266-255c6047435f admin admin] RBAC: Authorizing `identity:delete_project(project_id=63a2ceed7faa45fba9c6fbf4f03c2d9e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:31.960919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868edd6a-ec12-4735-a266-255c6047435f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:31.972261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1017/2054] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:31.972261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87722cf4-0a8a-4b30-82ac-ae9fdc3ba4ee None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:31.973499 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-87722cf4-0a8a-4b30-82ac-ae9fdc3ba4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:31.995157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:32.015616 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-868edd6a-ec12-4735-a266-255c6047435f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63a2ceed7faa45fba9c6fbf4f03c2d9e', 'request_id': 'req-868edd6a-ec12-4735-a266-255c6047435f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.021404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-868edd6a-ec12-4735-a266-255c6047435f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63a2ceed7faa45fba9c6fbf4f03c2d9e', 'request_id': 'req-868edd6a-ec12-4735-a266-255c6047435f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.028244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-868edd6a-ec12-4735-a266-255c6047435f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63a2ceed7faa45fba9c6fbf4f03c2d9e', 'request_id': 'req-868edd6a-ec12-4735-a266-255c6047435f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.037420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87722cf4-0a8a-4b30-82ac-ae9fdc3ba4ee None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:32.041545 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-868edd6a-ec12-4735-a266-255c6047435f admin admin] The token cache is being invalidate because project 63a2ceed7faa45fba9c6fbf4f03c2d9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:32.041837 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-868edd6a-ec12-4735-a266-255c6047435f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.042966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-868edd6a-ec12-4735-a266-255c6047435f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.047994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1038/2055] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:31 2022] DELETE /identity/v3/projects/63a2ceed7faa45fba9c6fbf4f03c2d9e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:32.047994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868edd6a-ec12-4735-a266-255c6047435f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:32.049204 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-868edd6a-ec12-4735-a266-255c6047435f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:32.097641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e72bae-0ff6-4133-a0a6-7b858d6e3b18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:32.098608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e72bae-0ff6-4133-a0a6-7b858d6e3b18 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:32.098900 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e72bae-0ff6-4133-a0a6-7b858d6e3b18 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:32.099182 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e72bae-0ff6-4133-a0a6-7b858d6e3b18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:32.101227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-35e72bae-0ff6-4133-a0a6-7b858d6e3b18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:32.107526 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:32.114435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868edd6a-ec12-4735-a266-255c6047435f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:32.119528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e93d5-cc5d-4ba6-a208-fd0a581148c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:32.120202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e93d5-cc5d-4ba6-a208-fd0a581148c0 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:32.120573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e93d5-cc5d-4ba6-a208-fd0a581148c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:32.120850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e93d5-cc5d-4ba6-a208-fd0a581148c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:32.122988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-026e93d5-cc5d-4ba6-a208-fd0a581148c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:32.166967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e72bae-0ff6-4133-a0a6-7b858d6e3b18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:32.168458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e72bae-0ff6-4133-a0a6-7b858d6e3b18 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:32.179640 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1018/2056] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:32.181037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1f1a3-609b-4515-bd6e-4a574a65a4a3 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:32.181569 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1f1a3-609b-4515-bd6e-4a574a65a4a3 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:32.181839 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1f1a3-609b-4515-bd6e-4a574a65a4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:32.203398 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f3c1f1a3-609b-4515-bd6e-4a574a65a4a3 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:32.204574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e93d5-cc5d-4ba6-a208-fd0a581148c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:32.204691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f3c1f1a3-609b-4515-bd6e-4a574a65a4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:32.206041 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e93d5-cc5d-4ba6-a208-fd0a581148c0 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:32.214423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1019/2057] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:32.219539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1039/2058] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:32.223719 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2003daa-4294-44e1-bde8-6f09ae3315d4 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:32.224044 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2003daa-4294-44e1-bde8-6f09ae3315d4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:32.224320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2003daa-4294-44e1-bde8-6f09ae3315d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:32.246113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f2003daa-4294-44e1-bde8-6f09ae3315d4 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:32.247702 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f2003daa-4294-44e1-bde8-6f09ae3315d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:32.260777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1020/2059] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:32.266567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e93d5-cc5d-4ba6-a208-fd0a581148c0 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:32.285468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e93d5-cc5d-4ba6-a208-fd0a581148c0 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:32.298040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba0dd6-86ed-49d8-9378-0cac531b9c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:32.298959 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba0dd6-86ed-49d8-9378-0cac531b9c75 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:32.299329 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba0dd6-86ed-49d8-9378-0cac531b9c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:32.300076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba0dd6-86ed-49d8-9378-0cac531b9c75 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:32.300928 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba0dd6-86ed-49d8-9378-0cac531b9c75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:32.301346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba0dd6-86ed-49d8-9378-0cac531b9c75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:32.303341 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba0dd6-86ed-49d8-9378-0cac531b9c75 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:32.322758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1040/2060] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:53:32.327118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2003daa-4294-44e1-bde8-6f09ae3315d4 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:32.339905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2003daa-4294-44e1-bde8-6f09ae3315d4 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:32.345355 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ecd8b-ece7-4aba-b34e-382d2f34eaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:32.345999 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ecd8b-ece7-4aba-b34e-382d2f34eaa9 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:32.346573 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ecd8b-ece7-4aba-b34e-382d2f34eaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:32.346573 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ecd8b-ece7-4aba-b34e-382d2f34eaa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:32.347264 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ecd8b-ece7-4aba-b34e-382d2f34eaa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:32.349546 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ecd8b-ece7-4aba-b34e-382d2f34eaa9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:32.391168 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1021/2061] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:32 2022] POST /identity/v3/projects => generated 359 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:32.396299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba0dd6-86ed-49d8-9378-0cac531b9c75 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:32.409306 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba0dd6-86ed-49d8-9378-0cac531b9c75 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:32.480958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588aebb3-a95b-4f1e-9241-2fa8a7f9270f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:32.481819 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aebb3-a95b-4f1e-9241-2fa8a7f9270f admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:32.482253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aebb3-a95b-4f1e-9241-2fa8a7f9270f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:32.482540 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588aebb3-a95b-4f1e-9241-2fa8a7f9270f admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:32.483210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588aebb3-a95b-4f1e-9241-2fa8a7f9270f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:32.485272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588aebb3-a95b-4f1e-9241-2fa8a7f9270f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:32.496387 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ecd8b-ece7-4aba-b34e-382d2f34eaa9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:32.498698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d48ecd8b-ece7-4aba-b34e-382d2f34eaa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:32.513979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1041/2062] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:32 2022] POST /identity/v3/users => generated 280 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:32.517343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ecd8b-ece7-4aba-b34e-382d2f34eaa9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:32.518316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588aebb3-a95b-4f1e-9241-2fa8a7f9270f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:32.522686 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:32.523341 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:32.523627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:32.523934 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9 admin admin] PATH_INFO: `/v3/projects/95eb40a1778a4e17ac5285847ece631d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:32.526250 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95eb40a1778a4e17ac5285847ece631d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:32.528246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:32.548402 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588aebb3-a95b-4f1e-9241-2fa8a7f9270f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:32.554222 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd167882-84bf-4826-bc16-9a24ddeaf57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:32.555019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd167882-84bf-4826-bc16-9a24ddeaf57a admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:32.555355 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd167882-84bf-4826-bc16-9a24ddeaf57a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:32.556145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd167882-84bf-4826-bc16-9a24ddeaf57a admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:32.557838 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd167882-84bf-4826-bc16-9a24ddeaf57a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:32.559853 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd167882-84bf-4826-bc16-9a24ddeaf57a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:32.574508 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1042/2063] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:32.578909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd167882-84bf-4826-bc16-9a24ddeaf57a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:32.586169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:32.589873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd167882-84bf-4826-bc16-9a24ddeaf57a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:32.595719 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95eb40a1778a4e17ac5285847ece631d', 'request_id': 'req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.604564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95eb40a1778a4e17ac5285847ece631d', 'request_id': 'req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.613580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95eb40a1778a4e17ac5285847ece631d', 'request_id': 'req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.626711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9 admin admin] The token cache is being invalidate because project 95eb40a1778a4e17ac5285847ece631d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:32.627004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.627671 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.629316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1022/2064] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:32 2022] DELETE /identity/v3/projects/95eb40a1778a4e17ac5285847ece631d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:32.633249 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:32.633249 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:32.657855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394311a-5364-4bd5-a286-857a806c9064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:32.658487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394311a-5364-4bd5-a286-857a806c9064 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:32.658848 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394311a-5364-4bd5-a286-857a806c9064 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:32.659122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394311a-5364-4bd5-a286-857a806c9064 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:32.659780 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394311a-5364-4bd5-a286-857a806c9064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:32.661389 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394311a-5364-4bd5-a286-857a806c9064 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:32.664079 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:32.677374 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1043/2065] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:32.678114 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394311a-5364-4bd5-a286-857a806c9064 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:32.680674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-6394311a-5364-4bd5-a286-857a806c9064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:32.694446 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394311a-5364-4bd5-a286-857a806c9064 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:32.699685 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc393a5-3c46-438d-87a4-8eefb75cb61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:32.700338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc393a5-3c46-438d-87a4-8eefb75cb61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:32.700618 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc393a5-3c46-438d-87a4-8eefb75cb61f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:32.700919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc393a5-3c46-438d-87a4-8eefb75cb61f admin admin] PATH_INFO: `/v3/users/78694aaa9a4d47809f35ac5826f9819b` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:32.701035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341af1c-50f2-4a1c-9a1b-c0d0cd4fcff9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:32.702055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc393a5-3c46-438d-87a4-8eefb75cb61f admin admin] RBAC: Authorizing `identity:delete_user(user_id=78694aaa9a4d47809f35ac5826f9819b)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:32.703994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc393a5-3c46-438d-87a4-8eefb75cb61f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:32.705762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58612c-9057-4d2d-b9bb-811e086d44ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:32.706432 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58612c-9057-4d2d-b9bb-811e086d44ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:32.706727 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58612c-9057-4d2d-b9bb-811e086d44ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:32.707107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58612c-9057-4d2d-b9bb-811e086d44ff admin admin] PATH_INFO: `/v3/projects/7dff918de1654017bf21ba8ef2b891c5` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:32.708265 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58612c-9057-4d2d-b9bb-811e086d44ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dff918de1654017bf21ba8ef2b891c5)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:32.710173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58612c-9057-4d2d-b9bb-811e086d44ff admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:32.754756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ebc393a5-3c46-438d-87a4-8eefb75cb61f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78694aaa9a4d47809f35ac5826f9819b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.763202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ebc393a5-3c46-438d-87a4-8eefb75cb61f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78694aaa9a4d47809f35ac5826f9819b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.764846 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0b58612c-9057-4d2d-b9bb-811e086d44ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dff918de1654017bf21ba8ef2b891c5', 'request_id': 'req-0b58612c-9057-4d2d-b9bb-811e086d44ff'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.771592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0b58612c-9057-4d2d-b9bb-811e086d44ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dff918de1654017bf21ba8ef2b891c5', 'request_id': 'req-0b58612c-9057-4d2d-b9bb-811e086d44ff'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.776380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ebc393a5-3c46-438d-87a4-8eefb75cb61f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78694aaa9a4d47809f35ac5826f9819b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.782689 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0b58612c-9057-4d2d-b9bb-811e086d44ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dff918de1654017bf21ba8ef2b891c5', 'request_id': 'req-0b58612c-9057-4d2d-b9bb-811e086d44ff'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.785449 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ebc393a5-3c46-438d-87a4-8eefb75cb61f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78694aaa9a4d47809f35ac5826f9819b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.786439 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ebc393a5-3c46-438d-87a4-8eefb75cb61f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78694aaa9a4d47809f35ac5826f9819b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.796207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0b58612c-9057-4d2d-b9bb-811e086d44ff admin admin] The token cache is being invalidate because project 7dff918de1654017bf21ba8ef2b891c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:32.796511 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0b58612c-9057-4d2d-b9bb-811e086d44ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.797207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-0b58612c-9057-4d2d-b9bb-811e086d44ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:32.798916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1023/2066] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:32 2022] DELETE /identity/v3/projects/7dff918de1654017bf21ba8ef2b891c5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:32.801096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58612c-9057-4d2d-b9bb-811e086d44ff admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:32.802633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0b58612c-9057-4d2d-b9bb-811e086d44ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:32.807704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1044/2067] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:32 2022] DELETE /identity/v3/users/78694aaa9a4d47809f35ac5826f9819b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:32.808786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc393a5-3c46-438d-87a4-8eefb75cb61f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:32.810559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ebc393a5-3c46-438d-87a4-8eefb75cb61f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:32.881933 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58612c-9057-4d2d-b9bb-811e086d44ff admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:32.887813 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:32.908867 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc393a5-3c46-438d-87a4-8eefb75cb61f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:32.914880 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d69229-ff7f-4799-8ed9-56bdfd9d6550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:32.915462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d69229-ff7f-4799-8ed9-56bdfd9d6550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:32.915744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d69229-ff7f-4799-8ed9-56bdfd9d6550 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:32.916182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d69229-ff7f-4799-8ed9-56bdfd9d6550 admin admin] PATH_INFO: `/v3/users/8b0755a8bf534c8e97543b65bed3ac3e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:32.917224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d69229-ff7f-4799-8ed9-56bdfd9d6550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b0755a8bf534c8e97543b65bed3ac3e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:32.919646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d69229-ff7f-4799-8ed9-56bdfd9d6550 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:32.981414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd6196-9487-4210-80ea-5aea7158acdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:32.982491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd6196-9487-4210-80ea-5aea7158acdf admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:32.982491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd6196-9487-4210-80ea-5aea7158acdf admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:32.982733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd6196-9487-4210-80ea-5aea7158acdf admin admin] PATH_INFO: `/v3/projects/039f94c315a54e7e956afdcf9e5c4917/users/9a918dde24b643f4a10018c01db42234/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:32.989354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-88d69229-ff7f-4799-8ed9-56bdfd9d6550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b0755a8bf534c8e97543b65bed3ac3e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.000681 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-88d69229-ff7f-4799-8ed9-56bdfd9d6550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b0755a8bf534c8e97543b65bed3ac3e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.001509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd6196-9487-4210-80ea-5aea7158acdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=039f94c315a54e7e956afdcf9e5c4917, user_id=9a918dde24b643f4a10018c01db42234, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:33.004329 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd6196-9487-4210-80ea-5aea7158acdf admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:33.024546 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1024/2068] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:32 2022] PUT /identity/v3/projects/039f94c315a54e7e956afdcf9e5c4917/users/9a918dde24b643f4a10018c01db42234/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:33.024904 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:33.027672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-88d69229-ff7f-4799-8ed9-56bdfd9d6550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b0755a8bf534c8e97543b65bed3ac3e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.037139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a79875-0a0e-470b-b958-08790a82a541 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:33.037559 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a79875-0a0e-470b-b958-08790a82a541 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:33.038463 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a79875-0a0e-470b-b958-08790a82a541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:33.040032 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-88d69229-ff7f-4799-8ed9-56bdfd9d6550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b0755a8bf534c8e97543b65bed3ac3e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.040973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-88d69229-ff7f-4799-8ed9-56bdfd9d6550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b0755a8bf534c8e97543b65bed3ac3e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.048427 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1045/2069] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:32 2022] DELETE /identity/v3/users/8b0755a8bf534c8e97543b65bed3ac3e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:33.073271 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d5a79875-0a0e-470b-b958-08790a82a541 None None] MFA Rules not processed for user `9a918dde24b643f4a10018c01db42234`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:33.074499 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d5a79875-0a0e-470b-b958-08790a82a541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:33.166108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1025/2070] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:33.205238 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d69229-ff7f-4799-8ed9-56bdfd9d6550 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:33.206724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-88d69229-ff7f-4799-8ed9-56bdfd9d6550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:33.212231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a79875-0a0e-470b-b958-08790a82a541 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:33.213844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d5a79875-0a0e-470b-b958-08790a82a541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:33.297617 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d69229-ff7f-4799-8ed9-56bdfd9d6550 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:33.304224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b31720-8f31-44be-ad32-6ee310a92f64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:33.304689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b31720-8f31-44be-ad32-6ee310a92f64 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:33.305662 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b31720-8f31-44be-ad32-6ee310a92f64 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:33.305992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b31720-8f31-44be-ad32-6ee310a92f64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:33.310498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a79875-0a0e-470b-b958-08790a82a541 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:33.315041 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8dbbf-af54-45b2-9145-3ff78f118876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:33.315978 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8dbbf-af54-45b2-9145-3ff78f118876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:33.316430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8dbbf-af54-45b2-9145-3ff78f118876 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:33.316670 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b31720-8f31-44be-ad32-6ee310a92f64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:33.316771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8dbbf-af54-45b2-9145-3ff78f118876 admin admin] PATH_INFO: `/v3/projects/e6821d623ea346e493189e0aed9f71f4` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:33.317980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8dbbf-af54-45b2-9145-3ff78f118876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6821d623ea346e493189e0aed9f71f4)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:33.318182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b31720-8f31-44be-ad32-6ee310a92f64 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:33.320237 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8dbbf-af54-45b2-9145-3ff78f118876 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:33.328638 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1046/2071] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:33 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:33.347697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b31720-8f31-44be-ad32-6ee310a92f64 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:33.348193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a7b31720-8f31-44be-ad32-6ee310a92f64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:33.359766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:33.367733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b31720-8f31-44be-ad32-6ee310a92f64 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:33.373465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c471456-d831-4da0-82d6-78ceb121bb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:33.374111 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c471456-d831-4da0-82d6-78ceb121bb5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:33.374451 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c471456-d831-4da0-82d6-78ceb121bb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:33.374666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c471456-d831-4da0-82d6-78ceb121bb5b admin admin] PATH_INFO: `/v3/users/9a918dde24b643f4a10018c01db42234` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:33.375949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c471456-d831-4da0-82d6-78ceb121bb5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a918dde24b643f4a10018c01db42234)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:33.377944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c471456-d831-4da0-82d6-78ceb121bb5b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:33.393595 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f5f8dbbf-af54-45b2-9145-3ff78f118876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6821d623ea346e493189e0aed9f71f4', 'request_id': 'req-f5f8dbbf-af54-45b2-9145-3ff78f118876'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.402818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f5f8dbbf-af54-45b2-9145-3ff78f118876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6821d623ea346e493189e0aed9f71f4', 'request_id': 'req-f5f8dbbf-af54-45b2-9145-3ff78f118876'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.414788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f5f8dbbf-af54-45b2-9145-3ff78f118876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6821d623ea346e493189e0aed9f71f4', 'request_id': 'req-f5f8dbbf-af54-45b2-9145-3ff78f118876'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.434627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f5f8dbbf-af54-45b2-9145-3ff78f118876 admin admin] The token cache is being invalidate because project e6821d623ea346e493189e0aed9f71f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:33.434981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f5f8dbbf-af54-45b2-9145-3ff78f118876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.435330 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2c471456-d831-4da0-82d6-78ceb121bb5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a918dde24b643f4a10018c01db42234'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.435620 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f5f8dbbf-af54-45b2-9145-3ff78f118876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.440676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1026/2072] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:33 2022] DELETE /identity/v3/projects/e6821d623ea346e493189e0aed9f71f4 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:33.442772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547db3d6-4d9b-4409-8ff3-5e2d034d587d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:33.443107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547db3d6-4d9b-4409-8ff3-5e2d034d587d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:33.443489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547db3d6-4d9b-4409-8ff3-5e2d034d587d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:33.452390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2c471456-d831-4da0-82d6-78ceb121bb5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a918dde24b643f4a10018c01db42234'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.468314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-547db3d6-4d9b-4409-8ff3-5e2d034d587d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:33.469575 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-547db3d6-4d9b-4409-8ff3-5e2d034d587d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:33.472707 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2c471456-d831-4da0-82d6-78ceb121bb5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a918dde24b643f4a10018c01db42234'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.482191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2c471456-d831-4da0-82d6-78ceb121bb5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a918dde24b643f4a10018c01db42234'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.483104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2c471456-d831-4da0-82d6-78ceb121bb5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a918dde24b643f4a10018c01db42234'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.489911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1047/2073] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:33 2022] DELETE /identity/v3/users/9a918dde24b643f4a10018c01db42234 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:33.490719 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c471456-d831-4da0-82d6-78ceb121bb5b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:33.492371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2c471456-d831-4da0-82d6-78ceb121bb5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:33.539289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1027/2074] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:53:33.540755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78f010-aa94-4825-81d3-3a7c582e0725 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:33.541040 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78f010-aa94-4825-81d3-3a7c582e0725 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:33.541311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78f010-aa94-4825-81d3-3a7c582e0725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:33.560720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ec78f010-aa94-4825-81d3-3a7c582e0725 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:33.561924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ec78f010-aa94-4825-81d3-3a7c582e0725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:33.567222 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c471456-d831-4da0-82d6-78ceb121bb5b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:33.570479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1028/2075] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:33.571450 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8666774-296e-4827-bd20-10362a04e222 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:33.571738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8666774-296e-4827-bd20-10362a04e222 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:33.572046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8666774-296e-4827-bd20-10362a04e222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:33.578694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eba8f-b698-4de3-bef7-eb7d76797899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:33.579625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eba8f-b698-4de3-bef7-eb7d76797899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:33.580496 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eba8f-b698-4de3-bef7-eb7d76797899 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:33.580496 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eba8f-b698-4de3-bef7-eb7d76797899 admin admin] PATH_INFO: `/v3/projects/1fdb9363081a4daba6c8d537b38ba589` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:33.581421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300eba8f-b698-4de3-bef7-eb7d76797899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fdb9363081a4daba6c8d537b38ba589)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:33.583345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300eba8f-b698-4de3-bef7-eb7d76797899 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:33.591098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:33.599414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d8666774-296e-4827-bd20-10362a04e222 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:33.600825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d8666774-296e-4827-bd20-10362a04e222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:33.609624 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1029/2076] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:33.610892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8666774-296e-4827-bd20-10362a04e222 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:33.612119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d8666774-296e-4827-bd20-10362a04e222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:33.645281 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-300eba8f-b698-4de3-bef7-eb7d76797899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fdb9363081a4daba6c8d537b38ba589', 'request_id': 'req-300eba8f-b698-4de3-bef7-eb7d76797899'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.652004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-300eba8f-b698-4de3-bef7-eb7d76797899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fdb9363081a4daba6c8d537b38ba589', 'request_id': 'req-300eba8f-b698-4de3-bef7-eb7d76797899'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.659031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-300eba8f-b698-4de3-bef7-eb7d76797899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fdb9363081a4daba6c8d537b38ba589', 'request_id': 'req-300eba8f-b698-4de3-bef7-eb7d76797899'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.673059 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-300eba8f-b698-4de3-bef7-eb7d76797899 admin admin] The token cache is being invalidate because project 1fdb9363081a4daba6c8d537b38ba589 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:33.673348 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-300eba8f-b698-4de3-bef7-eb7d76797899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.673995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-300eba8f-b698-4de3-bef7-eb7d76797899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:33.674927 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8666774-296e-4827-bd20-10362a04e222 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:33.675479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1048/2077] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:33 2022] DELETE /identity/v3/projects/1fdb9363081a4daba6c8d537b38ba589 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:33.676356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eba8f-b698-4de3-bef7-eb7d76797899 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:33.677627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-300eba8f-b698-4de3-bef7-eb7d76797899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:33.732822 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05db43-28bf-4b1e-9694-c7c18de52e86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:33.733438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05db43-28bf-4b1e-9694-c7c18de52e86 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:33.733724 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05db43-28bf-4b1e-9694-c7c18de52e86 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:33.734002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05db43-28bf-4b1e-9694-c7c18de52e86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:33.735757 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9b05db43-28bf-4b1e-9694-c7c18de52e86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:33.744849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eba8f-b698-4de3-bef7-eb7d76797899 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:33.745617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:33.748131 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05db43-28bf-4b1e-9694-c7c18de52e86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:33.749899 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05db43-28bf-4b1e-9694-c7c18de52e86 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:33.750797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c559f-3fec-4e0f-95de-4639a3afc080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:33.751409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c559f-3fec-4e0f-95de-4639a3afc080 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:33.751692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c559f-3fec-4e0f-95de-4639a3afc080 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:33.752006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c559f-3fec-4e0f-95de-4639a3afc080 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:33.752649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c559f-3fec-4e0f-95de-4639a3afc080 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:33.752952 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c559f-3fec-4e0f-95de-4639a3afc080 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:33.754531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c559f-3fec-4e0f-95de-4639a3afc080 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:33.762265 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1030/2078] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:33.767644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1049/2079] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:53:33.771766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c559f-3fec-4e0f-95de-4639a3afc080 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:33.772106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f168d4-e395-45d5-8a54-c5490a13a38d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:33.773091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f168d4-e395-45d5-8a54-c5490a13a38d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:33.773435 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f168d4-e395-45d5-8a54-c5490a13a38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:33.782929 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c559f-3fec-4e0f-95de-4639a3afc080 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:33.787593 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fd7ea-727f-45d8-acf2-7088c674a4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:33.788294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fd7ea-727f-45d8-acf2-7088c674a4ea admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:33.788572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fd7ea-727f-45d8-acf2-7088c674a4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:33.788841 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53fd7ea-727f-45d8-acf2-7088c674a4ea admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:33.789462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fd7ea-727f-45d8-acf2-7088c674a4ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:33.791346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53fd7ea-727f-45d8-acf2-7088c674a4ea admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:33.794255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e5f168d4-e395-45d5-8a54-c5490a13a38d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:33.795998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e5f168d4-e395-45d5-8a54-c5490a13a38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:33.811370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1031/2080] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:33.821216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1dfe2-e199-44f3-94a8-8c12eff0655b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:33.821819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1dfe2-e199-44f3-94a8-8c12eff0655b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:33.821819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1dfe2-e199-44f3-94a8-8c12eff0655b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:33.832052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1050/2081] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:33 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:33.838140 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fd7ea-727f-45d8-acf2-7088c674a4ea admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:33.849532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-70d1dfe2-e199-44f3-94a8-8c12eff0655b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:33.850767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-70d1dfe2-e199-44f3-94a8-8c12eff0655b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:33.936935 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53fd7ea-727f-45d8-acf2-7088c674a4ea admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:33.938176 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1032/2082] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:33.943452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1dfe2-e199-44f3-94a8-8c12eff0655b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:33.952268 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b6cd0-10ef-4fcd-a270-a350dc1f55ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:33.953722 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b6cd0-10ef-4fcd-a270-a350dc1f55ab admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:33.954226 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b6cd0-10ef-4fcd-a270-a350dc1f55ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:33.954727 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b6cd0-10ef-4fcd-a270-a350dc1f55ab admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:33.955446 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b6cd0-10ef-4fcd-a270-a350dc1f55ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:33.955896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1dfe2-e199-44f3-94a8-8c12eff0655b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:33.957956 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b6cd0-10ef-4fcd-a270-a350dc1f55ab admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:33.962467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f32c43-1f46-4c2a-a259-1c2dd1b9d21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:33.963645 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f32c43-1f46-4c2a-a259-1c2dd1b9d21f admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:33.964091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f32c43-1f46-4c2a-a259-1c2dd1b9d21f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:33.964360 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f32c43-1f46-4c2a-a259-1c2dd1b9d21f admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:33.965218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f32c43-1f46-4c2a-a259-1c2dd1b9d21f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:33.965607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f32c43-1f46-4c2a-a259-1c2dd1b9d21f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:33.970554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f32c43-1f46-4c2a-a259-1c2dd1b9d21f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:33.971807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:33.992343 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1051/2083] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:33 2022] POST /identity/v3/users => generated 290 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:33.993394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b6cd0-10ef-4fcd-a270-a350dc1f55ab admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:33.996397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1033/2084] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:53:34.001257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f32c43-1f46-4c2a-a259-1c2dd1b9d21f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.008177 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b6cd0-10ef-4fcd-a270-a350dc1f55ab admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.013924 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb60cbb-73f4-4188-a33d-ec0fd390ee0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.014631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb60cbb-73f4-4188-a33d-ec0fd390ee0e admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.015050 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb60cbb-73f4-4188-a33d-ec0fd390ee0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.015330 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb60cbb-73f4-4188-a33d-ec0fd390ee0e admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.015649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f32c43-1f46-4c2a-a259-1c2dd1b9d21f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.016122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb60cbb-73f4-4188-a33d-ec0fd390ee0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.017740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb60cbb-73f4-4188-a33d-ec0fd390ee0e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.021190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37affd51-3320-4411-86f0-11b0b3527558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.021793 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37affd51-3320-4411-86f0-11b0b3527558 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.022066 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37affd51-3320-4411-86f0-11b0b3527558 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.022380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37affd51-3320-4411-86f0-11b0b3527558 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.023335 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37affd51-3320-4411-86f0-11b0b3527558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.025257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37affd51-3320-4411-86f0-11b0b3527558 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.030504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1052/2085] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:34.034444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb60cbb-73f4-4188-a33d-ec0fd390ee0e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.044776 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb60cbb-73f4-4188-a33d-ec0fd390ee0e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.050037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab64354-404e-459e-9b16-32567325f064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.050735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab64354-404e-459e-9b16-32567325f064 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.050982 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab64354-404e-459e-9b16-32567325f064 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.051242 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab64354-404e-459e-9b16-32567325f064 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.051908 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab64354-404e-459e-9b16-32567325f064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.054090 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab64354-404e-459e-9b16-32567325f064 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.062336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1034/2086] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:34 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:34.066654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1053/2087] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:53:34.067112 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37affd51-3320-4411-86f0-11b0b3527558 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.071284 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab64354-404e-459e-9b16-32567325f064 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.083377 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab64354-404e-459e-9b16-32567325f064 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.154273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b463141-9ad7-49ce-aeca-1aa639ecd778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.154944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b463141-9ad7-49ce-aeca-1aa639ecd778 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.155224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b463141-9ad7-49ce-aeca-1aa639ecd778 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.155495 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b463141-9ad7-49ce-aeca-1aa639ecd778 admin admin] PATH_INFO: `/v3/projects/76968f7a5ce14ded939c71b66a62e579/users/28077b5d9a4641b8bdacf0a1b111fac4/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.156259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37affd51-3320-4411-86f0-11b0b3527558 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.161447 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2366607-33c2-469a-a289-ed48372969e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.162137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2366607-33c2-469a-a289-ed48372969e7 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.162490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2366607-33c2-469a-a289-ed48372969e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.163422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2366607-33c2-469a-a289-ed48372969e7 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.164530 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2366607-33c2-469a-a289-ed48372969e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.167133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2366607-33c2-469a-a289-ed48372969e7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.173163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b463141-9ad7-49ce-aeca-1aa639ecd778 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76968f7a5ce14ded939c71b66a62e579, user_id=28077b5d9a4641b8bdacf0a1b111fac4, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.176430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b463141-9ad7-49ce-aeca-1aa639ecd778 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.192825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1035/2088] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:34 2022] POST /identity/v3/users => generated 284 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:34.193974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2366607-33c2-469a-a289-ed48372969e7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.195793 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c2366607-33c2-469a-a289-ed48372969e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:34.199977 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1054/2089] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:34 2022] PUT /identity/v3/projects/76968f7a5ce14ded939c71b66a62e579/users/28077b5d9a4641b8bdacf0a1b111fac4/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:34.199977 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b463141-9ad7-49ce-aeca-1aa639ecd778 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.210086 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2366607-33c2-469a-a289-ed48372969e7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.255585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:34.276791 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c482ba-3872-49c6-8fb2-c944355041a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.277472 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c482ba-3872-49c6-8fb2-c944355041a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.277763 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c482ba-3872-49c6-8fb2-c944355041a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.278205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c482ba-3872-49c6-8fb2-c944355041a5 admin admin] PATH_INFO: `/v3/projects/039f94c315a54e7e956afdcf9e5c4917` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.279362 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c482ba-3872-49c6-8fb2-c944355041a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=039f94c315a54e7e956afdcf9e5c4917)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.281345 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c482ba-3872-49c6-8fb2-c944355041a5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.283509 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b463141-9ad7-49ce-aeca-1aa639ecd778 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.288897 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029bf25-7883-457c-9700-4b20a9712537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.289516 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029bf25-7883-457c-9700-4b20a9712537 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.289799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029bf25-7883-457c-9700-4b20a9712537 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.290097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029bf25-7883-457c-9700-4b20a9712537 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.290739 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029bf25-7883-457c-9700-4b20a9712537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.292573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e029bf25-7883-457c-9700-4b20a9712537 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.303166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1055/2090] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:34.305146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b5504-e8b6-4751-8e8d-37239984c4ab None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.305429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b5504-e8b6-4751-8e8d-37239984c4ab None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.305705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b5504-e8b6-4751-8e8d-37239984c4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.338527 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4c3b5504-e8b6-4751-8e8d-37239984c4ab None None] MFA Rules not processed for user `28077b5d9a4641b8bdacf0a1b111fac4`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:34.339791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4c3b5504-e8b6-4751-8e8d-37239984c4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:34.347423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-65c482ba-3872-49c6-8fb2-c944355041a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '039f94c315a54e7e956afdcf9e5c4917', 'request_id': 'req-65c482ba-3872-49c6-8fb2-c944355041a5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:34.356082 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-65c482ba-3872-49c6-8fb2-c944355041a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '039f94c315a54e7e956afdcf9e5c4917', 'request_id': 'req-65c482ba-3872-49c6-8fb2-c944355041a5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:34.365285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-65c482ba-3872-49c6-8fb2-c944355041a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '039f94c315a54e7e956afdcf9e5c4917', 'request_id': 'req-65c482ba-3872-49c6-8fb2-c944355041a5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:34.377708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-65c482ba-3872-49c6-8fb2-c944355041a5 admin admin] The token cache is being invalidate because project 039f94c315a54e7e956afdcf9e5c4917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:34.378003 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-65c482ba-3872-49c6-8fb2-c944355041a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:34.378680 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-65c482ba-3872-49c6-8fb2-c944355041a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:34.380425 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1036/2091] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:34 2022] DELETE /identity/v3/projects/039f94c315a54e7e956afdcf9e5c4917 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:34.381137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c482ba-3872-49c6-8fb2-c944355041a5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.382485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-65c482ba-3872-49c6-8fb2-c944355041a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:34.391413 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:34.417603 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1056/2092] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:34.418886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b39df-05a3-44ba-ae2b-b92c1b51b7b7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.419174 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b39df-05a3-44ba-ae2b-b92c1b51b7b7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.419450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b39df-05a3-44ba-ae2b-b92c1b51b7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.440833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d36b39df-05a3-44ba-ae2b-b92c1b51b7b7 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:34.442057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d36b39df-05a3-44ba-ae2b-b92c1b51b7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:34.450479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1057/2093] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:34.451258 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b39df-05a3-44ba-ae2b-b92c1b51b7b7 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.452565 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d36b39df-05a3-44ba-ae2b-b92c1b51b7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:34.453805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c482ba-3872-49c6-8fb2-c944355041a5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.458825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180989e-17c0-403a-9fdd-51d4108934cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.459545 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180989e-17c0-403a-9fdd-51d4108934cf admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.459759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180989e-17c0-403a-9fdd-51d4108934cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.460060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180989e-17c0-403a-9fdd-51d4108934cf admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.460737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180989e-17c0-403a-9fdd-51d4108934cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.462364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180989e-17c0-403a-9fdd-51d4108934cf admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.466174 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b39df-05a3-44ba-ae2b-b92c1b51b7b7 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.471349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9cf2c-355c-43d8-b4c4-08d98cf89de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.471981 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9cf2c-355c-43d8-b4c4-08d98cf89de5 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.472262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9cf2c-355c-43d8-b4c4-08d98cf89de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.472429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1037/2094] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:34.472544 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9cf2c-355c-43d8-b4c4-08d98cf89de5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.473142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9cf2c-355c-43d8-b4c4-08d98cf89de5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.473647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacab442-92a7-40c0-89da-e9d0c109c8bb None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.473905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacab442-92a7-40c0-89da-e9d0c109c8bb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.474176 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacab442-92a7-40c0-89da-e9d0c109c8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.475054 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9cf2c-355c-43d8-b4c4-08d98cf89de5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.494831 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-cacab442-92a7-40c0-89da-e9d0c109c8bb None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:34.496062 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cacab442-92a7-40c0-89da-e9d0c109c8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:34.504551 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1038/2095] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:34.505501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacab442-92a7-40c0-89da-e9d0c109c8bb None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.515418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1058/2096] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:34 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:34.515418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9cf2c-355c-43d8-b4c4-08d98cf89de5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.515706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacab442-92a7-40c0-89da-e9d0c109c8bb None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.573961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f42fd-738f-4455-8430-76a72af8e65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.574647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f42fd-738f-4455-8430-76a72af8e65d admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.574929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f42fd-738f-4455-8430-76a72af8e65d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.575197 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f42fd-738f-4455-8430-76a72af8e65d admin admin] PATH_INFO: `/v3/projects/d1b7b4531372417d9b9929a823938a7a/users/35cb7bfbb282465db8b32f36f4dae016/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.578563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9cf2c-355c-43d8-b4c4-08d98cf89de5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.582960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:34.583276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd036f3-5f8d-4f79-a4b8-0794df50cb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.583904 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd036f3-5f8d-4f79-a4b8-0794df50cb2f admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.584245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd036f3-5f8d-4f79-a4b8-0794df50cb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.584562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd036f3-5f8d-4f79-a4b8-0794df50cb2f admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.586248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd036f3-5f8d-4f79-a4b8-0794df50cb2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:34.586556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd036f3-5f8d-4f79-a4b8-0794df50cb2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.588624 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd036f3-5f8d-4f79-a4b8-0794df50cb2f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.591061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f42fd-738f-4455-8430-76a72af8e65d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1b7b4531372417d9b9929a823938a7a, user_id=35cb7bfbb282465db8b32f36f4dae016, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.594181 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f42fd-738f-4455-8430-76a72af8e65d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.602523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1059/2097] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:53:34.603332 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd036f3-5f8d-4f79-a4b8-0794df50cb2f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.608746 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1039/2098] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:34 2022] PUT /identity/v3/projects/d1b7b4531372417d9b9929a823938a7a/users/35cb7bfbb282465db8b32f36f4dae016/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:34.609571 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f42fd-738f-4455-8430-76a72af8e65d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.619023 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd036f3-5f8d-4f79-a4b8-0794df50cb2f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.686649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aeed82-2555-43ac-909f-0415ea531227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.687562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f42fd-738f-4455-8430-76a72af8e65d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.687691 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeed82-2555-43ac-909f-0415ea531227 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.687691 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeed82-2555-43ac-909f-0415ea531227 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.688074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeed82-2555-43ac-909f-0415ea531227 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.688541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aeed82-2555-43ac-909f-0415ea531227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.690559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aeed82-2555-43ac-909f-0415ea531227 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.692484 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a5664-f6ae-404a-bb3a-be359e8b5184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.693098 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a5664-f6ae-404a-bb3a-be359e8b5184 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.693406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a5664-f6ae-404a-bb3a-be359e8b5184 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.693683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a5664-f6ae-404a-bb3a-be359e8b5184 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.694338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a5664-f6ae-404a-bb3a-be359e8b5184 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.696285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a5664-f6ae-404a-bb3a-be359e8b5184 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.713093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1060/2099] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:34 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:34.714352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81639d-9183-4850-99a6-e779607a2238 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.714641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81639d-9183-4850-99a6-e779607a2238 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.714919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81639d-9183-4850-99a6-e779607a2238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.731627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1040/2100] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:34 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:34.731627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a5664-f6ae-404a-bb3a-be359e8b5184 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.745688 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5a81639d-9183-4850-99a6-e779607a2238 None None] MFA Rules not processed for user `35cb7bfbb282465db8b32f36f4dae016`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:34.746912 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5a81639d-9183-4850-99a6-e779607a2238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:34.774292 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:34.792675 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a5664-f6ae-404a-bb3a-be359e8b5184 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.797416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199eebb8-f888-4b35-97f3-3895f43a59ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.798059 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199eebb8-f888-4b35-97f3-3895f43a59ab admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.798343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199eebb8-f888-4b35-97f3-3895f43a59ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.798617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199eebb8-f888-4b35-97f3-3895f43a59ab admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.799289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199eebb8-f888-4b35-97f3-3895f43a59ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.800992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199eebb8-f888-4b35-97f3-3895f43a59ab admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.815756 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1041/2101] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:34.815756 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199eebb8-f888-4b35-97f3-3895f43a59ab admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.816000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1061/2102] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:34.818921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81639d-9183-4850-99a6-e779607a2238 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.824621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199eebb8-f888-4b35-97f3-3895f43a59ab admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.829427 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81639d-9183-4850-99a6-e779607a2238 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.829759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d688466-c3a4-4837-b7a9-26561201f9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.830490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d688466-c3a4-4837-b7a9-26561201f9f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.830803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d688466-c3a4-4837-b7a9-26561201f9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.831074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d688466-c3a4-4837-b7a9-26561201f9f4 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.831691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d688466-c3a4-4837-b7a9-26561201f9f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.833630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d688466-c3a4-4837-b7a9-26561201f9f4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.834377 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1562823-d93d-4827-a398-af755df30c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.835576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1562823-d93d-4827-a398-af755df30c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.835879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1562823-d93d-4827-a398-af755df30c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.836158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1562823-d93d-4827-a398-af755df30c3d admin admin] PATH_INFO: `/v3/projects/b98a16b41c694a88889cf26910054d70/users/3fdd32cf293d48ee8556c3d296740a72/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.853803 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1562823-d93d-4827-a398-af755df30c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b98a16b41c694a88889cf26910054d70, user_id=3fdd32cf293d48ee8556c3d296740a72, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.855541 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1042/2103] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:34 2022] POST /identity/v3/users => generated 288 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:34.856452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d688466-c3a4-4837-b7a9-26561201f9f4 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.857302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1562823-d93d-4827-a398-af755df30c3d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.867720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d688466-c3a4-4837-b7a9-26561201f9f4 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.875005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1062/2104] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:34 2022] PUT /identity/v3/projects/b98a16b41c694a88889cf26910054d70/users/3fdd32cf293d48ee8556c3d296740a72/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:34.875005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1562823-d93d-4827-a398-af755df30c3d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.922879 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:34.950628 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e19a8-3923-43a3-963a-0ea93a9bc017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.951260 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e19a8-3923-43a3-963a-0ea93a9bc017 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.951529 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e19a8-3923-43a3-963a-0ea93a9bc017 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.951871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e19a8-3923-43a3-963a-0ea93a9bc017 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.952473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e19a8-3923-43a3-963a-0ea93a9bc017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.955010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1562823-d93d-4827-a398-af755df30c3d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:34.957714 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e19a8-3923-43a3-963a-0ea93a9bc017 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.963753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca8a08-0f47-47d0-931a-98e1b2369103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:34.964737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca8a08-0f47-47d0-931a-98e1b2369103 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:34.965171 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca8a08-0f47-47d0-931a-98e1b2369103 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:34.965448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca8a08-0f47-47d0-931a-98e1b2369103 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:34.966098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca8a08-0f47-47d0-931a-98e1b2369103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:34.967761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca8a08-0f47-47d0-931a-98e1b2369103 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:34.983766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1063/2105] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:34.985075 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca8a08-0f47-47d0-931a-98e1b2369103 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.991771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1043/2106] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:34 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:34.994961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e19a8-3923-43a3-963a-0ea93a9bc017 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:34.996095 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca8a08-0f47-47d0-931a-98e1b2369103 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:35.055011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe6514-dbec-48b2-b206-e2628d4a52c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:35.055621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe6514-dbec-48b2-b206-e2628d4a52c6 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.055730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e19a8-3923-43a3-963a-0ea93a9bc017 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:35.055979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe6514-dbec-48b2-b206-e2628d4a52c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.056519 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe6514-dbec-48b2-b206-e2628d4a52c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:35.057158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe6514-dbec-48b2-b206-e2628d4a52c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:35.058755 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe6514-dbec-48b2-b206-e2628d4a52c6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:35.060863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a38cc-404c-4a62-b986-39555c9b3c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:35.061462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a38cc-404c-4a62-b986-39555c9b3c84 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.061738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a38cc-404c-4a62-b986-39555c9b3c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.062011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a38cc-404c-4a62-b986-39555c9b3c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:35.062640 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a38cc-404c-4a62-b986-39555c9b3c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:35.064320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a38cc-404c-4a62-b986-39555c9b3c84 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:35.068758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1064/2107] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:35.075915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe6514-dbec-48b2-b206-e2628d4a52c6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:35.076090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1044/2108] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:35.077285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a38cc-404c-4a62-b986-39555c9b3c84 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:35.085530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe6514-dbec-48b2-b206-e2628d4a52c6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:35.088097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a38cc-404c-4a62-b986-39555c9b3c84 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:35.090824 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f5da3-8e01-45dc-9d4e-6706331cd983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:35.091471 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f5da3-8e01-45dc-9d4e-6706331cd983 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.092341 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f5da3-8e01-45dc-9d4e-6706331cd983 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.092341 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f5da3-8e01-45dc-9d4e-6706331cd983 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:35.092674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f5da3-8e01-45dc-9d4e-6706331cd983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:35.093145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d04ee-d5f1-4cba-aa45-01b2198dc0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:35.093789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d04ee-d5f1-4cba-aa45-01b2198dc0ee admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.094063 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d04ee-d5f1-4cba-aa45-01b2198dc0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.094429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d04ee-d5f1-4cba-aa45-01b2198dc0ee admin admin] PATH_INFO: `/v3/projects/b98a16b41c694a88889cf26910054d70/users/3fdd32cf293d48ee8556c3d296740a72/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:35.094618 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f5da3-8e01-45dc-9d4e-6706331cd983 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:35.096394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d04ee-d5f1-4cba-aa45-01b2198dc0ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b98a16b41c694a88889cf26910054d70, user_id=3fdd32cf293d48ee8556c3d296740a72, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:35.099015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d04ee-d5f1-4cba-aa45-01b2198dc0ee admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:35.109816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1045/2109] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:35 2022] PUT /identity/v3/projects/b98a16b41c694a88889cf26910054d70/users/3fdd32cf293d48ee8556c3d296740a72/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:35.110791 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d04ee-d5f1-4cba-aa45-01b2198dc0ee admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:35.124709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1065/2110] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:35 2022] POST /identity/v3/users => generated 289 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:35.126245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967d21b-90b6-4195-ad87-3a2e189cbd0f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.126535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967d21b-90b6-4195-ad87-3a2e189cbd0f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.126851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967d21b-90b6-4195-ad87-3a2e189cbd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:35.154998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4967d21b-90b6-4195-ad87-3a2e189cbd0f None None] MFA Rules not processed for user `3fdd32cf293d48ee8556c3d296740a72`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:35.156280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4967d21b-90b6-4195-ad87-3a2e189cbd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:35.161400 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:35.183592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d04ee-d5f1-4cba-aa45-01b2198dc0ee admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:35.188463 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee5c60-e2da-4269-a5fe-46f86ac08be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:35.189763 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee5c60-e2da-4269-a5fe-46f86ac08be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.190215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee5c60-e2da-4269-a5fe-46f86ac08be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.190427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee5c60-e2da-4269-a5fe-46f86ac08be7 admin admin] PATH_INFO: `/v3/projects/a10aea007c2749cbb46faa241fdcf148/users/48b6760f4a664979a5dbfd10b27ec8c0/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:35.206072 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee5c60-e2da-4269-a5fe-46f86ac08be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a10aea007c2749cbb46faa241fdcf148, user_id=48b6760f4a664979a5dbfd10b27ec8c0, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:35.209687 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee5c60-e2da-4269-a5fe-46f86ac08be7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:35.223841 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1046/2111] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:35 2022] PUT /identity/v3/projects/a10aea007c2749cbb46faa241fdcf148/users/48b6760f4a664979a5dbfd10b27ec8c0/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:35.225103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee5c60-e2da-4269-a5fe-46f86ac08be7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:35.230162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1066/2112] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:35.235125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b567d12-7d52-426b-8f54-4be1aa2883ab None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.235387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b567d12-7d52-426b-8f54-4be1aa2883ab None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.235683 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b567d12-7d52-426b-8f54-4be1aa2883ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:35.265394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0b567d12-7d52-426b-8f54-4be1aa2883ab None None] MFA Rules not processed for user `48b6760f4a664979a5dbfd10b27ec8c0`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:35.266605 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0b567d12-7d52-426b-8f54-4be1aa2883ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:35.285609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:35.309295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee5c60-e2da-4269-a5fe-46f86ac08be7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:35.314172 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d0101-3e94-4373-a241-8fde40e15685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:35.314783 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d0101-3e94-4373-a241-8fde40e15685 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.315066 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d0101-3e94-4373-a241-8fde40e15685 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.315341 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d0101-3e94-4373-a241-8fde40e15685 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:35.316063 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d0101-3e94-4373-a241-8fde40e15685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:35.317617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d0101-3e94-4373-a241-8fde40e15685 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:35.328626 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1047/2113] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:35.329442 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d0101-3e94-4373-a241-8fde40e15685 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:35.331220 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-861d0101-3e94-4373-a241-8fde40e15685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:35.339653 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1067/2114] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:35.340560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b567d12-7d52-426b-8f54-4be1aa2883ab None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:35.353782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b567d12-7d52-426b-8f54-4be1aa2883ab None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:35.362230 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c432a3-31d8-43a6-b723-b25cedfbb650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:35.362930 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c432a3-31d8-43a6-b723-b25cedfbb650 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.363197 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c432a3-31d8-43a6-b723-b25cedfbb650 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.363474 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c432a3-31d8-43a6-b723-b25cedfbb650 admin admin] PATH_INFO: `/v3/projects/9eab55b5372c4eeca21d5f2e986479a9/users/92f7b76f15ac4c1f810480acce986237/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:35.383751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c432a3-31d8-43a6-b723-b25cedfbb650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eab55b5372c4eeca21d5f2e986479a9, user_id=92f7b76f15ac4c1f810480acce986237, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:35.386777 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c432a3-31d8-43a6-b723-b25cedfbb650 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:35.392661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d0101-3e94-4373-a241-8fde40e15685 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:35.397048 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c0804-4dac-4ff6-a150-b2f212741182 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:35.397837 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c0804-4dac-4ff6-a150-b2f212741182 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.398121 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c0804-4dac-4ff6-a150-b2f212741182 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.398396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c0804-4dac-4ff6-a150-b2f212741182 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:35.405387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1068/2115] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:35 2022] PUT /identity/v3/projects/9eab55b5372c4eeca21d5f2e986479a9/users/92f7b76f15ac4c1f810480acce986237/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:35.405387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c432a3-31d8-43a6-b723-b25cedfbb650 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:35.407540 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d0c432a3-31d8-43a6-b723-b25cedfbb650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:35.472084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c0804-4dac-4ff6-a150-b2f212741182 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:35.473512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c432a3-31d8-43a6-b723-b25cedfbb650 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:35.474070 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c0804-4dac-4ff6-a150-b2f212741182 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:35.478043 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118f676-de20-4547-b5b5-9c5fca6ad53d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:35.478664 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118f676-de20-4547-b5b5-9c5fca6ad53d service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.478947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118f676-de20-4547-b5b5-9c5fca6ad53d service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.479222 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118f676-de20-4547-b5b5-9c5fca6ad53d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:35.486312 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1048/2116] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:35.487527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c0804-4dac-4ff6-a150-b2f212741182 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:35.499258 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c0804-4dac-4ff6-a150-b2f212741182 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:35.502414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:35.567148 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48f588-b4bd-42dd-b099-82777ab95327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:35.568083 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48f588-b4bd-42dd-b099-82777ab95327 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.568250 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48f588-b4bd-42dd-b099-82777ab95327 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.568524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48f588-b4bd-42dd-b099-82777ab95327 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:35.569775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48f588-b4bd-42dd-b099-82777ab95327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:35.572333 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b48f588-b4bd-42dd-b099-82777ab95327 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:35.575032 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118f676-de20-4547-b5b5-9c5fca6ad53d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:35.576938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3118f676-de20-4547-b5b5-9c5fca6ad53d service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:35.588001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1049/2117] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:35.588117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48f588-b4bd-42dd-b099-82777ab95327 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:35.590563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1069/2118] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:35.600859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b48f588-b4bd-42dd-b099-82777ab95327 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:35.607043 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7c710-db2c-46d0-b01a-4285b9468657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:35.607719 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7c710-db2c-46d0-b01a-4285b9468657 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.608040 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7c710-db2c-46d0-b01a-4285b9468657 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.608363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7c710-db2c-46d0-b01a-4285b9468657 admin admin] PATH_INFO: `/v3/projects/9eab55b5372c4eeca21d5f2e986479a9/users/92f7b76f15ac4c1f810480acce986237/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:35.610866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7c710-db2c-46d0-b01a-4285b9468657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eab55b5372c4eeca21d5f2e986479a9, user_id=92f7b76f15ac4c1f810480acce986237, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:35.614081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7c710-db2c-46d0-b01a-4285b9468657 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:35.626367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1050/2119] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:35 2022] PUT /identity/v3/projects/9eab55b5372c4eeca21d5f2e986479a9/users/92f7b76f15ac4c1f810480acce986237/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:35.636996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73863571-fa5e-4cb8-9a5e-4e2c4dfae6f8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.637283 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73863571-fa5e-4cb8-9a5e-4e2c4dfae6f8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.637552 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73863571-fa5e-4cb8-9a5e-4e2c4dfae6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:35.668817 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-73863571-fa5e-4cb8-9a5e-4e2c4dfae6f8 None None] MFA Rules not processed for user `92f7b76f15ac4c1f810480acce986237`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:35.670029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-73863571-fa5e-4cb8-9a5e-4e2c4dfae6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:35.750634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1070/2120] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:35.763221 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7c710-db2c-46d0-b01a-4285b9468657 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:35.777578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:35.826380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7c710-db2c-46d0-b01a-4285b9468657 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:35.830164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c80026-1d7a-4227-a688-4c3eb33dffe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:35.830776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c80026-1d7a-4227-a688-4c3eb33dffe3 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.831084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c80026-1d7a-4227-a688-4c3eb33dffe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.831389 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c80026-1d7a-4227-a688-4c3eb33dffe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:35.841175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c80026-1d7a-4227-a688-4c3eb33dffe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:35.842655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c80026-1d7a-4227-a688-4c3eb33dffe3 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:35.852987 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1051/2121] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:35 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:35.971645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73863571-fa5e-4cb8-9a5e-4e2c4dfae6f8 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:35.990613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73863571-fa5e-4cb8-9a5e-4e2c4dfae6f8 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:35.996560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f92861-6957-44e7-89a0-50f2ebce4392 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:35.998344 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f92861-6957-44e7-89a0-50f2ebce4392 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:35.998726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f92861-6957-44e7-89a0-50f2ebce4392 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:35.999087 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f92861-6957-44e7-89a0-50f2ebce4392 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:36.066785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f92861-6957-44e7-89a0-50f2ebce4392 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:36.068876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f92861-6957-44e7-89a0-50f2ebce4392 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:36.081900 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1071/2122] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:36.484298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c80026-1d7a-4227-a688-4c3eb33dffe3 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:36.554836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c80026-1d7a-4227-a688-4c3eb33dffe3 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:36.560527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021870f-e2c7-4c65-8077-e97cbc04725f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:36.561163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021870f-e2c7-4c65-8077-e97cbc04725f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:36.561437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021870f-e2c7-4c65-8077-e97cbc04725f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:36.561710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021870f-e2c7-4c65-8077-e97cbc04725f admin admin] PATH_INFO: `/v3/users/28077b5d9a4641b8bdacf0a1b111fac4` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:36.562817 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021870f-e2c7-4c65-8077-e97cbc04725f admin admin] RBAC: Authorizing `identity:delete_user(user_id=28077b5d9a4641b8bdacf0a1b111fac4)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:36.564770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021870f-e2c7-4c65-8077-e97cbc04725f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:36.614151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f021870f-e2c7-4c65-8077-e97cbc04725f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28077b5d9a4641b8bdacf0a1b111fac4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:36.621553 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f021870f-e2c7-4c65-8077-e97cbc04725f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28077b5d9a4641b8bdacf0a1b111fac4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:36.635234 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f021870f-e2c7-4c65-8077-e97cbc04725f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28077b5d9a4641b8bdacf0a1b111fac4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:36.645403 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f021870f-e2c7-4c65-8077-e97cbc04725f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28077b5d9a4641b8bdacf0a1b111fac4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:36.646038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f021870f-e2c7-4c65-8077-e97cbc04725f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28077b5d9a4641b8bdacf0a1b111fac4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:36.652978 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1052/2123] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:36 2022] DELETE /identity/v3/users/28077b5d9a4641b8bdacf0a1b111fac4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:36.657435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724fe57-dbc4-465d-9a07-65d8d738d809 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:36.657724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724fe57-dbc4-465d-9a07-65d8d738d809 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:36.658029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724fe57-dbc4-465d-9a07-65d8d738d809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:36.678074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2724fe57-dbc4-465d-9a07-65d8d738d809 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:36.679316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2724fe57-dbc4-465d-9a07-65d8d738d809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:36.725200 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:36.742967 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1072/2124] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:36.752640 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021870f-e2c7-4c65-8077-e97cbc04725f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:36.754090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f021870f-e2c7-4c65-8077-e97cbc04725f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:36.790941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:36.814555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021870f-e2c7-4c65-8077-e97cbc04725f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:36.818247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cef24-d005-4ed8-892c-0d1329df8199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:36.818859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cef24-d005-4ed8-892c-0d1329df8199 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:36.819760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cef24-d005-4ed8-892c-0d1329df8199 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:36.820072 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cef24-d005-4ed8-892c-0d1329df8199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:36.829167 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6cef24-d005-4ed8-892c-0d1329df8199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:36.830602 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6cef24-d005-4ed8-892c-0d1329df8199 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:36.839948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1053/2125] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:37.112805 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724fe57-dbc4-465d-9a07-65d8d738d809 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:37.114362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2724fe57-dbc4-465d-9a07-65d8d738d809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:37.133436 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724fe57-dbc4-465d-9a07-65d8d738d809 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:37.139016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:37.139758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:37.140018 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:37.140298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3 admin admin] PATH_INFO: `/v3/projects/76968f7a5ce14ded939c71b66a62e579` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:37.141470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76968f7a5ce14ded939c71b66a62e579)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:37.143762 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:37.213994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76968f7a5ce14ded939c71b66a62e579', 'request_id': 'req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:37.220891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76968f7a5ce14ded939c71b66a62e579', 'request_id': 'req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:37.228430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76968f7a5ce14ded939c71b66a62e579', 'request_id': 'req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:37.245799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3 admin admin] The token cache is being invalidate because project 76968f7a5ce14ded939c71b66a62e579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:37.246077 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:37.246948 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:37.248782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1073/2126] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:37 2022] DELETE /identity/v3/projects/76968f7a5ce14ded939c71b66a62e579 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:37.252390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cef24-d005-4ed8-892c-0d1329df8199 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:37.262489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6cef24-d005-4ed8-892c-0d1329df8199 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:37.332435 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc16a1fd-a8c1-4ca2-a9ab-2155c56096d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:37.333069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16a1fd-a8c1-4ca2-a9ab-2155c56096d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:37.333349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16a1fd-a8c1-4ca2-a9ab-2155c56096d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:37.333619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc16a1fd-a8c1-4ca2-a9ab-2155c56096d7 admin admin] PATH_INFO: `/v3/users/3fdd32cf293d48ee8556c3d296740a72` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:37.334778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16a1fd-a8c1-4ca2-a9ab-2155c56096d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fdd32cf293d48ee8556c3d296740a72)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:37.338041 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc16a1fd-a8c1-4ca2-a9ab-2155c56096d7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:37.405677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc16a1fd-a8c1-4ca2-a9ab-2155c56096d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fdd32cf293d48ee8556c3d296740a72'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:37.416664 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc16a1fd-a8c1-4ca2-a9ab-2155c56096d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fdd32cf293d48ee8556c3d296740a72'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:37.436278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc16a1fd-a8c1-4ca2-a9ab-2155c56096d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fdd32cf293d48ee8556c3d296740a72'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:37.446230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc16a1fd-a8c1-4ca2-a9ab-2155c56096d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fdd32cf293d48ee8556c3d296740a72'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:37.446903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fc16a1fd-a8c1-4ca2-a9ab-2155c56096d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fdd32cf293d48ee8556c3d296740a72'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:37.454494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1054/2127] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:37 2022] DELETE /identity/v3/users/3fdd32cf293d48ee8556c3d296740a72 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:37.775151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:37.776949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:37.810164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4aa3a-32f1-4b31-bbb8-3757378859b3 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:37.869787 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d04e44-c16f-4f37-9b79-a9ad91a28875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:37.870430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d04e44-c16f-4f37-9b79-a9ad91a28875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:37.870712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d04e44-c16f-4f37-9b79-a9ad91a28875 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:37.870990 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d04e44-c16f-4f37-9b79-a9ad91a28875 admin admin] PATH_INFO: `/v3/projects/b98a16b41c694a88889cf26910054d70` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:37.872158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d04e44-c16f-4f37-9b79-a9ad91a28875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b98a16b41c694a88889cf26910054d70)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:37.874194 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d04e44-c16f-4f37-9b79-a9ad91a28875 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:37.925340 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-73d04e44-c16f-4f37-9b79-a9ad91a28875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b98a16b41c694a88889cf26910054d70', 'request_id': 'req-73d04e44-c16f-4f37-9b79-a9ad91a28875'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:37.932031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-73d04e44-c16f-4f37-9b79-a9ad91a28875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b98a16b41c694a88889cf26910054d70', 'request_id': 'req-73d04e44-c16f-4f37-9b79-a9ad91a28875'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:37.938285 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-73d04e44-c16f-4f37-9b79-a9ad91a28875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b98a16b41c694a88889cf26910054d70', 'request_id': 'req-73d04e44-c16f-4f37-9b79-a9ad91a28875'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:37.947308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:37.952104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-73d04e44-c16f-4f37-9b79-a9ad91a28875 admin admin] The token cache is being invalidate because project b98a16b41c694a88889cf26910054d70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:37.952407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-73d04e44-c16f-4f37-9b79-a9ad91a28875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:37.953318 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-73d04e44-c16f-4f37-9b79-a9ad91a28875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:37.955560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1074/2128] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:37 2022] DELETE /identity/v3/projects/b98a16b41c694a88889cf26910054d70 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:37.961314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0159642-ef54-4e87-ae77-e67f7edff1c5 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:37.961773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0159642-ef54-4e87-ae77-e67f7edff1c5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:37.961872 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0159642-ef54-4e87-ae77-e67f7edff1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:37.982159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b0159642-ef54-4e87-ae77-e67f7edff1c5 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:37.983373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b0159642-ef54-4e87-ae77-e67f7edff1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:38.031198 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:38.044319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1055/2129] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:38.052260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e8947-72b1-4995-a00b-1b1c65195f0d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:38.052527 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e8947-72b1-4995-a00b-1b1c65195f0d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:38.052803 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e8947-72b1-4995-a00b-1b1c65195f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:38.073026 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-780e8947-72b1-4995-a00b-1b1c65195f0d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:38.074223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-780e8947-72b1-4995-a00b-1b1c65195f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:38.082492 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1075/2130] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:38.086786 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0159642-ef54-4e87-ae77-e67f7edff1c5 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:38.096681 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0159642-ef54-4e87-ae77-e67f7edff1c5 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:38.101857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087801cf-ccc3-4e68-bf95-bcb8460420a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:38.102474 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087801cf-ccc3-4e68-bf95-bcb8460420a1 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:38.102757 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087801cf-ccc3-4e68-bf95-bcb8460420a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:38.103032 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087801cf-ccc3-4e68-bf95-bcb8460420a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:38.103673 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087801cf-ccc3-4e68-bf95-bcb8460420a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:38.104009 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087801cf-ccc3-4e68-bf95-bcb8460420a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:38.105636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087801cf-ccc3-4e68-bf95-bcb8460420a1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:38.118164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1056/2131] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:53:38.122512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e8947-72b1-4995-a00b-1b1c65195f0d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:38.131845 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e8947-72b1-4995-a00b-1b1c65195f0d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:38.136589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe48d2-ec5f-4033-b16f-e37534667cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:38.137182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe48d2-ec5f-4033-b16f-e37534667cff admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:38.137614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe48d2-ec5f-4033-b16f-e37534667cff admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:38.137888 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe48d2-ec5f-4033-b16f-e37534667cff admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:38.138507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe48d2-ec5f-4033-b16f-e37534667cff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:38.140450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe48d2-ec5f-4033-b16f-e37534667cff admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:38.168920 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1076/2132] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:38 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:38.173016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087801cf-ccc3-4e68-bf95-bcb8460420a1 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:38.184133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087801cf-ccc3-4e68-bf95-bcb8460420a1 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:38.248485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33f302-96f8-4604-8586-9847ce40d6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:38.251543 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33f302-96f8-4604-8586-9847ce40d6df admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:38.251845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33f302-96f8-4604-8586-9847ce40d6df admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:38.252141 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f33f302-96f8-4604-8586-9847ce40d6df admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:38.252800 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33f302-96f8-4604-8586-9847ce40d6df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:38.256435 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f33f302-96f8-4604-8586-9847ce40d6df admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:38.282024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1057/2133] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:38 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:38.283122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe48d2-ec5f-4033-b16f-e37534667cff admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:38.294445 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe48d2-ec5f-4033-b16f-e37534667cff admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:38.299214 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016ddff-2cbf-4da3-8d5b-76061938b504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:38.299846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016ddff-2cbf-4da3-8d5b-76061938b504 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:38.300133 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016ddff-2cbf-4da3-8d5b-76061938b504 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:38.300410 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016ddff-2cbf-4da3-8d5b-76061938b504 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:38.301181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016ddff-2cbf-4da3-8d5b-76061938b504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:38.302781 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016ddff-2cbf-4da3-8d5b-76061938b504 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:38.314588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1077/2134] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:53:38.318796 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33f302-96f8-4604-8586-9847ce40d6df admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:38.329128 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f33f302-96f8-4604-8586-9847ce40d6df admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:38.333788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1642497-ce98-4d98-877d-f5402db65616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:38.334386 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1642497-ce98-4d98-877d-f5402db65616 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:38.335249 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1642497-ce98-4d98-877d-f5402db65616 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:38.335528 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1642497-ce98-4d98-877d-f5402db65616 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:38.336199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1642497-ce98-4d98-877d-f5402db65616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:38.337820 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1642497-ce98-4d98-877d-f5402db65616 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:38.348832 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1058/2135] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:38.353587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016ddff-2cbf-4da3-8d5b-76061938b504 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:38.364378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016ddff-2cbf-4da3-8d5b-76061938b504 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:38.369648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f12d82-0575-44ca-8fee-80506fce74f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:38.369831 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f12d82-0575-44ca-8fee-80506fce74f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:38.370063 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f12d82-0575-44ca-8fee-80506fce74f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:38.370336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f12d82-0575-44ca-8fee-80506fce74f8 admin admin] PATH_INFO: `/v3/projects/d2fa7f6a4fa74be080063529af301998/users/0d2c0ed843ed41b080896a043817f738/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:38.385144 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f12d82-0575-44ca-8fee-80506fce74f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2fa7f6a4fa74be080063529af301998, user_id=0d2c0ed843ed41b080896a043817f738, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:38.387790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f12d82-0575-44ca-8fee-80506fce74f8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:38.401145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1078/2136] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:38 2022] PUT /identity/v3/projects/d2fa7f6a4fa74be080063529af301998/users/0d2c0ed843ed41b080896a043817f738/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:38.412753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a33247-fa6f-4b72-9843-27542a7d9561 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:38.413648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a33247-fa6f-4b72-9843-27542a7d9561 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:38.413934 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a33247-fa6f-4b72-9843-27542a7d9561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:38.443118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f0a33247-fa6f-4b72-9843-27542a7d9561 None None] MFA Rules not processed for user `0d2c0ed843ed41b080896a043817f738`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:38.444407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f0a33247-fa6f-4b72-9843-27542a7d9561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:38.509012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1059/2137] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:38.520348 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f12d82-0575-44ca-8fee-80506fce74f8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:38.522504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f9f12d82-0575-44ca-8fee-80506fce74f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:38.584153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f12d82-0575-44ca-8fee-80506fce74f8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:38.588336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ee724-5bbf-42dd-ba68-9425bdb60eac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:38.588947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ee724-5bbf-42dd-ba68-9425bdb60eac service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:38.589228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ee724-5bbf-42dd-ba68-9425bdb60eac service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:38.589500 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ee724-5bbf-42dd-ba68-9425bdb60eac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:38.597865 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ee724-5bbf-42dd-ba68-9425bdb60eac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:38.599301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ee724-5bbf-42dd-ba68-9425bdb60eac service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:38.609484 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1079/2138] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:38.635067 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a33247-fa6f-4b72-9843-27542a7d9561 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:38.695790 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:38.709008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a33247-fa6f-4b72-9843-27542a7d9561 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:38.714090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fd890-41af-4aa0-95fd-da556da213e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:38.714751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fd890-41af-4aa0-95fd-da556da213e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:38.715029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fd890-41af-4aa0-95fd-da556da213e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:38.715300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fd890-41af-4aa0-95fd-da556da213e9 admin admin] PATH_INFO: `/v3/users/0d2c0ed843ed41b080896a043817f738` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:38.716443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fd890-41af-4aa0-95fd-da556da213e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d2c0ed843ed41b080896a043817f738)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:38.718369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fd890-41af-4aa0-95fd-da556da213e9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:38.766555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-655fd890-41af-4aa0-95fd-da556da213e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d2c0ed843ed41b080896a043817f738'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:38.775695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-655fd890-41af-4aa0-95fd-da556da213e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d2c0ed843ed41b080896a043817f738'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:38.790768 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-655fd890-41af-4aa0-95fd-da556da213e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d2c0ed843ed41b080896a043817f738'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:38.798532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-655fd890-41af-4aa0-95fd-da556da213e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d2c0ed843ed41b080896a043817f738'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:38.799504 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-655fd890-41af-4aa0-95fd-da556da213e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d2c0ed843ed41b080896a043817f738'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:38.805844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1060/2139] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:38 2022] DELETE /identity/v3/users/0d2c0ed843ed41b080896a043817f738 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:38.810705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a58a2f-8e35-44ca-8977-bfe2242c5507 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:38.810994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a58a2f-8e35-44ca-8977-bfe2242c5507 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:38.811265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a58a2f-8e35-44ca-8977-bfe2242c5507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:38.832148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b4a58a2f-8e35-44ca-8977-bfe2242c5507 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:38.833599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b4a58a2f-8e35-44ca-8977-bfe2242c5507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:38.901808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1080/2140] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:38.912143 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fd890-41af-4aa0-95fd-da556da213e9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:38.913723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-655fd890-41af-4aa0-95fd-da556da213e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:38.989630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fd890-41af-4aa0-95fd-da556da213e9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:38.993402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c679e-510d-4ad7-a20d-0aaf18c80b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:38.994013 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c679e-510d-4ad7-a20d-0aaf18c80b56 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:38.994467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c679e-510d-4ad7-a20d-0aaf18c80b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:38.994739 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c679e-510d-4ad7-a20d-0aaf18c80b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:39.004348 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c679e-510d-4ad7-a20d-0aaf18c80b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:39.005827 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c679e-510d-4ad7-a20d-0aaf18c80b56 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:39.016256 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1061/2141] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:39.288042 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a58a2f-8e35-44ca-8977-bfe2242c5507 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:39.289984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b4a58a2f-8e35-44ca-8977-bfe2242c5507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:39.315244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a58a2f-8e35-44ca-8977-bfe2242c5507 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:39.330132 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc110d7-e07f-4972-a623-870b0cb52046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:39.332284 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc110d7-e07f-4972-a623-870b0cb52046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:39.332284 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc110d7-e07f-4972-a623-870b0cb52046 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:39.332680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc110d7-e07f-4972-a623-870b0cb52046 admin admin] PATH_INFO: `/v3/projects/d2fa7f6a4fa74be080063529af301998` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:39.334356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc110d7-e07f-4972-a623-870b0cb52046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2fa7f6a4fa74be080063529af301998)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:39.336351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc110d7-e07f-4972-a623-870b0cb52046 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:39.385823 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:39.397484 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4fc110d7-e07f-4972-a623-870b0cb52046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2fa7f6a4fa74be080063529af301998', 'request_id': 'req-4fc110d7-e07f-4972-a623-870b0cb52046'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:39.403370 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4fc110d7-e07f-4972-a623-870b0cb52046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2fa7f6a4fa74be080063529af301998', 'request_id': 'req-4fc110d7-e07f-4972-a623-870b0cb52046'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:39.410342 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4fc110d7-e07f-4972-a623-870b0cb52046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2fa7f6a4fa74be080063529af301998', 'request_id': 'req-4fc110d7-e07f-4972-a623-870b0cb52046'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:39.429207 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4fc110d7-e07f-4972-a623-870b0cb52046 admin admin] The token cache is being invalidate because project d2fa7f6a4fa74be080063529af301998 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:39.429603 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4fc110d7-e07f-4972-a623-870b0cb52046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:39.430360 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4fc110d7-e07f-4972-a623-870b0cb52046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:39.432680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1081/2142] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:39 2022] DELETE /identity/v3/projects/d2fa7f6a4fa74be080063529af301998 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:39.439676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ebb83-dd76-4fd6-be72-e40430879bef None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:39.439998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ebb83-dd76-4fd6-be72-e40430879bef None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:39.440284 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ebb83-dd76-4fd6-be72-e40430879bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:39.464952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e99ebb83-dd76-4fd6-be72-e40430879bef None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:39.466195 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e99ebb83-dd76-4fd6-be72-e40430879bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:39.529946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1062/2143] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:39.538290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10730b-b16a-475c-92f3-994a0b74e69f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:39.538579 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10730b-b16a-475c-92f3-994a0b74e69f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:39.538854 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10730b-b16a-475c-92f3-994a0b74e69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:39.559650 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-fc10730b-b16a-475c-92f3-994a0b74e69f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:39.561019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fc10730b-b16a-475c-92f3-994a0b74e69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:39.569016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1082/2144] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:39.573417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ebb83-dd76-4fd6-be72-e40430879bef None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:39.583207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ebb83-dd76-4fd6-be72-e40430879bef None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:39.587975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99a7d5-4f66-4e1d-bd0a-61044fcfa052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:39.588580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99a7d5-4f66-4e1d-bd0a-61044fcfa052 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:39.588863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99a7d5-4f66-4e1d-bd0a-61044fcfa052 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:39.589137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99a7d5-4f66-4e1d-bd0a-61044fcfa052 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:39.589778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99a7d5-4f66-4e1d-bd0a-61044fcfa052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:39.590077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99a7d5-4f66-4e1d-bd0a-61044fcfa052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:39.591681 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99a7d5-4f66-4e1d-bd0a-61044fcfa052 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:39.602173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:39.606542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1063/2145] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:53:39.610433 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10730b-b16a-475c-92f3-994a0b74e69f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:39.620798 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10730b-b16a-475c-92f3-994a0b74e69f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:39.625758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee1c38-1a90-443c-9dfc-75b7807dfe79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:39.626363 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee1c38-1a90-443c-9dfc-75b7807dfe79 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:39.626638 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee1c38-1a90-443c-9dfc-75b7807dfe79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:39.626942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee1c38-1a90-443c-9dfc-75b7807dfe79 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:39.627597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee1c38-1a90-443c-9dfc-75b7807dfe79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:39.629755 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee1c38-1a90-443c-9dfc-75b7807dfe79 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:39.663867 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1083/2146] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:39 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:39.665912 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99a7d5-4f66-4e1d-bd0a-61044fcfa052 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:39.675855 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99a7d5-4f66-4e1d-bd0a-61044fcfa052 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:39.731980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b84e8-c9ec-4a97-824c-bb89105216c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:39.732584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b84e8-c9ec-4a97-824c-bb89105216c1 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:39.732861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b84e8-c9ec-4a97-824c-bb89105216c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:39.733159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b84e8-c9ec-4a97-824c-bb89105216c1 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:39.733786 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b84e8-c9ec-4a97-824c-bb89105216c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:39.735731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b84e8-c9ec-4a97-824c-bb89105216c1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:39.757183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1064/2147] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:39 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:39.761335 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee1c38-1a90-443c-9dfc-75b7807dfe79 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:39.771525 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee1c38-1a90-443c-9dfc-75b7807dfe79 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:39.776151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70737373-38cb-406b-a3aa-8a412044a508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:39.776742 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70737373-38cb-406b-a3aa-8a412044a508 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:39.777028 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70737373-38cb-406b-a3aa-8a412044a508 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:39.777297 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70737373-38cb-406b-a3aa-8a412044a508 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:39.777925 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70737373-38cb-406b-a3aa-8a412044a508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:39.779499 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70737373-38cb-406b-a3aa-8a412044a508 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:39.793276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1084/2148] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:53:39.794552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b84e8-c9ec-4a97-824c-bb89105216c1 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:39.805867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b84e8-c9ec-4a97-824c-bb89105216c1 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:39.810807 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f9809-e83a-4558-8bb5-8f8e7167c520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:39.811450 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f9809-e83a-4558-8bb5-8f8e7167c520 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:39.811689 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f9809-e83a-4558-8bb5-8f8e7167c520 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:39.812028 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f9809-e83a-4558-8bb5-8f8e7167c520 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:39.812764 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f9809-e83a-4558-8bb5-8f8e7167c520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:39.814403 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f9809-e83a-4558-8bb5-8f8e7167c520 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:39.833157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1065/2149] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:39.834214 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70737373-38cb-406b-a3aa-8a412044a508 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:39.845627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70737373-38cb-406b-a3aa-8a412044a508 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:39.850931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdaa67-3106-4ac7-94ae-2bd031332a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:39.851592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdaa67-3106-4ac7-94ae-2bd031332a07 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:39.851908 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdaa67-3106-4ac7-94ae-2bd031332a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:39.852190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcdaa67-3106-4ac7-94ae-2bd031332a07 admin admin] PATH_INFO: `/v3/projects/5a6e2209a3a3487a9d6a7255cbe56b63/users/218c699b6ddd43a4bcf9b842c59445f5/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:39.867270 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcdaa67-3106-4ac7-94ae-2bd031332a07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6e2209a3a3487a9d6a7255cbe56b63, user_id=218c699b6ddd43a4bcf9b842c59445f5, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:39.869944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcdaa67-3106-4ac7-94ae-2bd031332a07 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:39.886203 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1085/2150] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:39 2022] PUT /identity/v3/projects/5a6e2209a3a3487a9d6a7255cbe56b63/users/218c699b6ddd43a4bcf9b842c59445f5/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:39.892431 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50f82e-20a8-40a6-a580-0b53a66f607c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:39.892716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50f82e-20a8-40a6-a580-0b53a66f607c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:39.893035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50f82e-20a8-40a6-a580-0b53a66f607c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:39.914061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-6e50f82e-20a8-40a6-a580-0b53a66f607c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:39.915346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6e50f82e-20a8-40a6-a580-0b53a66f607c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:39.979372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1066/2151] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:39.989018 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdaa67-3106-4ac7-94ae-2bd031332a07 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:39.990754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cdcdaa67-3106-4ac7-94ae-2bd031332a07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:40.055019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcdaa67-3106-4ac7-94ae-2bd031332a07 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:40.059227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57ebe6-e126-4719-b647-bc313bc93a64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:40.059919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ebe6-e126-4719-b647-bc313bc93a64 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:40.060206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ebe6-e126-4719-b647-bc313bc93a64 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:40.060483 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57ebe6-e126-4719-b647-bc313bc93a64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:40.069782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57ebe6-e126-4719-b647-bc313bc93a64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:40.071215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57ebe6-e126-4719-b647-bc313bc93a64 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:40.082129 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1086/2152] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:41.078320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50f82e-20a8-40a6-a580-0b53a66f607c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:41.080635 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6e50f82e-20a8-40a6-a580-0b53a66f607c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:41.101488 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50f82e-20a8-40a6-a580-0b53a66f607c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:41.107255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ec818-862f-4305-bbb1-c8ccf8661b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:41.108233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ec818-862f-4305-bbb1-c8ccf8661b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:41.108659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ec818-862f-4305-bbb1-c8ccf8661b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:41.108814 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ec818-862f-4305-bbb1-c8ccf8661b96 admin admin] PATH_INFO: `/v3/users/48b6760f4a664979a5dbfd10b27ec8c0` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:41.110605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ec818-862f-4305-bbb1-c8ccf8661b96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48b6760f4a664979a5dbfd10b27ec8c0)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:41.112059 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ec818-862f-4305-bbb1-c8ccf8661b96 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:41.202007 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1f9ec818-862f-4305-bbb1-c8ccf8661b96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48b6760f4a664979a5dbfd10b27ec8c0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:41.214533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1f9ec818-862f-4305-bbb1-c8ccf8661b96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48b6760f4a664979a5dbfd10b27ec8c0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:41.228676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1f9ec818-862f-4305-bbb1-c8ccf8661b96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48b6760f4a664979a5dbfd10b27ec8c0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:41.230355 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:41.238900 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1f9ec818-862f-4305-bbb1-c8ccf8661b96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48b6760f4a664979a5dbfd10b27ec8c0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:41.239723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1f9ec818-862f-4305-bbb1-c8ccf8661b96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48b6760f4a664979a5dbfd10b27ec8c0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:41.250983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1067/2153] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:41 2022] DELETE /identity/v3/users/48b6760f4a664979a5dbfd10b27ec8c0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:41.254038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d4301-66d9-4168-9ee3-d6e7d3ffcef9 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:41.254532 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d4301-66d9-4168-9ee3-d6e7d3ffcef9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:41.254791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d4301-66d9-4168-9ee3-d6e7d3ffcef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:41.277528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-050d4301-66d9-4168-9ee3-d6e7d3ffcef9 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:41.278756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-050d4301-66d9-4168-9ee3-d6e7d3ffcef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:41.405141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1087/2154] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:41.418558 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ec818-862f-4305-bbb1-c8ccf8661b96 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:41.420394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1f9ec818-862f-4305-bbb1-c8ccf8661b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:41.500454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ec818-862f-4305-bbb1-c8ccf8661b96 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:41.504906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af89eba-8279-44f3-b419-941f333658ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:41.505604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af89eba-8279-44f3-b419-941f333658ab service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:41.505940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af89eba-8279-44f3-b419-941f333658ab service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:41.506217 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af89eba-8279-44f3-b419-941f333658ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:41.517347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af89eba-8279-44f3-b419-941f333658ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:41.519365 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af89eba-8279-44f3-b419-941f333658ab service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:41.540857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1068/2155] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:41.917352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d4301-66d9-4168-9ee3-d6e7d3ffcef9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:41.919784 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-050d4301-66d9-4168-9ee3-d6e7d3ffcef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:41.932078 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:41.953026 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d4301-66d9-4168-9ee3-d6e7d3ffcef9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:41.965454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b4ddb-fcb2-40a2-b042-81583aec31e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:41.966205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b4ddb-fcb2-40a2-b042-81583aec31e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:41.966486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b4ddb-fcb2-40a2-b042-81583aec31e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:41.967560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b4ddb-fcb2-40a2-b042-81583aec31e4 admin admin] PATH_INFO: `/v3/projects/a10aea007c2749cbb46faa241fdcf148` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:41.971167 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b4ddb-fcb2-40a2-b042-81583aec31e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a10aea007c2749cbb46faa241fdcf148)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:41.976142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b4ddb-fcb2-40a2-b042-81583aec31e4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:42.142386 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-984b4ddb-fcb2-40a2-b042-81583aec31e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a10aea007c2749cbb46faa241fdcf148', 'request_id': 'req-984b4ddb-fcb2-40a2-b042-81583aec31e4'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:42.175267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-984b4ddb-fcb2-40a2-b042-81583aec31e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a10aea007c2749cbb46faa241fdcf148', 'request_id': 'req-984b4ddb-fcb2-40a2-b042-81583aec31e4'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:42.188518 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-984b4ddb-fcb2-40a2-b042-81583aec31e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a10aea007c2749cbb46faa241fdcf148', 'request_id': 'req-984b4ddb-fcb2-40a2-b042-81583aec31e4'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:42.230581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-984b4ddb-fcb2-40a2-b042-81583aec31e4 admin admin] The token cache is being invalidate because project a10aea007c2749cbb46faa241fdcf148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:42.232300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-984b4ddb-fcb2-40a2-b042-81583aec31e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:42.234817 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-984b4ddb-fcb2-40a2-b042-81583aec31e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:42.248490 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1088/2156] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:41 2022] DELETE /identity/v3/projects/a10aea007c2749cbb46faa241fdcf148 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:42.253741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc4799-7210-419d-b6e7-fc6b3cd2e47f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:42.254293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc4799-7210-419d-b6e7-fc6b3cd2e47f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:42.254479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc4799-7210-419d-b6e7-fc6b3cd2e47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:42.293076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-30bc4799-7210-419d-b6e7-fc6b3cd2e47f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:42.294463 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-30bc4799-7210-419d-b6e7-fc6b3cd2e47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:42.521210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1069/2157] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:42.534885 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92656caf-266c-4bfc-8aea-b9422c5787f3 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:42.535348 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92656caf-266c-4bfc-8aea-b9422c5787f3 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:42.535427 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92656caf-266c-4bfc-8aea-b9422c5787f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:42.587371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-92656caf-266c-4bfc-8aea-b9422c5787f3 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:42.589450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-92656caf-266c-4bfc-8aea-b9422c5787f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:42.601260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1089/2158] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:42.605021 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc4799-7210-419d-b6e7-fc6b3cd2e47f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:42.626270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc4799-7210-419d-b6e7-fc6b3cd2e47f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:42.638612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2d018-45f2-4268-9f0f-6d0bca568233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:42.640428 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2d018-45f2-4268-9f0f-6d0bca568233 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:42.640716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2d018-45f2-4268-9f0f-6d0bca568233 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:42.641015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2d018-45f2-4268-9f0f-6d0bca568233 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:42.642867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2d018-45f2-4268-9f0f-6d0bca568233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:42.643489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2d018-45f2-4268-9f0f-6d0bca568233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:42.644891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2d018-45f2-4268-9f0f-6d0bca568233 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:42.698236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1070/2159] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:53:42.703323 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92656caf-266c-4bfc-8aea-b9422c5787f3 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:42.724949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92656caf-266c-4bfc-8aea-b9422c5787f3 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:42.732272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85bbcf-b4cb-4145-8e26-a31f02dc23bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:42.733245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85bbcf-b4cb-4145-8e26-a31f02dc23bc admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:42.733679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85bbcf-b4cb-4145-8e26-a31f02dc23bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:42.734092 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85bbcf-b4cb-4145-8e26-a31f02dc23bc admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:42.734879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85bbcf-b4cb-4145-8e26-a31f02dc23bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:42.737395 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85bbcf-b4cb-4145-8e26-a31f02dc23bc admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:42.812538 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1090/2160] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:42 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:42.816956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2d018-45f2-4268-9f0f-6d0bca568233 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:42.830140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2d018-45f2-4268-9f0f-6d0bca568233 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:42.990262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b871195-f2ed-47e8-a921-f3f89903dfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:42.991160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b871195-f2ed-47e8-a921-f3f89903dfcd admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:42.991656 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b871195-f2ed-47e8-a921-f3f89903dfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:42.992503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b871195-f2ed-47e8-a921-f3f89903dfcd admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:42.993267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b871195-f2ed-47e8-a921-f3f89903dfcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:42.995800 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b871195-f2ed-47e8-a921-f3f89903dfcd admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:43.052739 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85bbcf-b4cb-4145-8e26-a31f02dc23bc admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:43.057934 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1071/2161] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:42 2022] POST /identity/v3/users => generated 295 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:43.071450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85bbcf-b4cb-4145-8e26-a31f02dc23bc admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:43.105951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e69ef-fe1b-4239-a484-1ee320f23086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:43.112340 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e69ef-fe1b-4239-a484-1ee320f23086 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:43.112569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e69ef-fe1b-4239-a484-1ee320f23086 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:43.112851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e69ef-fe1b-4239-a484-1ee320f23086 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:43.113893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e69ef-fe1b-4239-a484-1ee320f23086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:43.115790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e69ef-fe1b-4239-a484-1ee320f23086 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:43.148085 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b871195-f2ed-47e8-a921-f3f89903dfcd admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:43.154704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1091/2162] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:43.162061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b871195-f2ed-47e8-a921-f3f89903dfcd admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:43.177631 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91decb-9690-46ed-9325-e9a2bef49b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:43.178630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91decb-9690-46ed-9325-e9a2bef49b49 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:43.179127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91decb-9690-46ed-9325-e9a2bef49b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:43.179602 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91decb-9690-46ed-9325-e9a2bef49b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:43.180495 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91decb-9690-46ed-9325-e9a2bef49b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:43.182481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91decb-9690-46ed-9325-e9a2bef49b49 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:43.195935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:43.214817 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e69ef-fe1b-4239-a484-1ee320f23086 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:43.225507 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1072/2163] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:43.229404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e69ef-fe1b-4239-a484-1ee320f23086 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:43.247600 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7f33f-570c-455d-afbb-83499607a716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:43.248573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7f33f-570c-455d-afbb-83499607a716 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:43.248931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7f33f-570c-455d-afbb-83499607a716 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:43.249298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7f33f-570c-455d-afbb-83499607a716 admin admin] PATH_INFO: `/v3/projects/ab18ed20453340a59ab3732de6483034/users/0cceaddf5fb045f5a3e2de8e3c0be216/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:43.304361 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7f33f-570c-455d-afbb-83499607a716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab18ed20453340a59ab3732de6483034, user_id=0cceaddf5fb045f5a3e2de8e3c0be216, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:43.307569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7f33f-570c-455d-afbb-83499607a716 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:43.349106 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1092/2164] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:43 2022] PUT /identity/v3/projects/ab18ed20453340a59ab3732de6483034/users/0cceaddf5fb045f5a3e2de8e3c0be216/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:43.358349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b024fe-5974-46d3-910b-482ec995d997 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:43.359305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b024fe-5974-46d3-910b-482ec995d997 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:43.359735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b024fe-5974-46d3-910b-482ec995d997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:43.452876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-33b024fe-5974-46d3-910b-482ec995d997 None None] MFA Rules not processed for user `0cceaddf5fb045f5a3e2de8e3c0be216`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:43.454947 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-33b024fe-5974-46d3-910b-482ec995d997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:43.641865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1073/2165] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:43.655727 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7f33f-570c-455d-afbb-83499607a716 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:43.658237 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-76f7f33f-570c-455d-afbb-83499607a716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:43.826955 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7f33f-570c-455d-afbb-83499607a716 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:43.831812 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a7a00-486f-4455-8150-b977ba09b86d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:43.832675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a7a00-486f-4455-8150-b977ba09b86d service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:43.833053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a7a00-486f-4455-8150-b977ba09b86d service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:43.833487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5a7a00-486f-4455-8150-b977ba09b86d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:43.863224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a7a00-486f-4455-8150-b977ba09b86d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:43.865150 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5a7a00-486f-4455-8150-b977ba09b86d service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:43.892908 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1093/2166] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:46.180597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e608d2-64c0-4d6f-8db2-d5a209ce4db9 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:46.181153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e608d2-64c0-4d6f-8db2-d5a209ce4db9 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:46.181527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e608d2-64c0-4d6f-8db2-d5a209ce4db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:46.219045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-61e608d2-64c0-4d6f-8db2-d5a209ce4db9 None None] MFA Rules not processed for user `218c699b6ddd43a4bcf9b842c59445f5`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:46.220296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-61e608d2-64c0-4d6f-8db2-d5a209ce4db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:46.294379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1074/2167] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:46.297255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a7a00-486f-4455-8150-b977ba09b86d service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:46.298782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4a5a7a00-486f-4455-8150-b977ba09b86d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:46.337120 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:46.380331 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5a7a00-486f-4455-8150-b977ba09b86d service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:46.384982 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d1850-d22f-418a-8973-5caf7646887b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:46.385619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d1850-d22f-418a-8973-5caf7646887b admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:46.385895 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d1850-d22f-418a-8973-5caf7646887b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:46.386162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d1850-d22f-418a-8973-5caf7646887b admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:46.386794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d1850-d22f-418a-8973-5caf7646887b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:46.388769 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d1850-d22f-418a-8973-5caf7646887b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:46.418082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1094/2168] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:46 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:53:46.421632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e608d2-64c0-4d6f-8db2-d5a209ce4db9 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:46.436935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e608d2-64c0-4d6f-8db2-d5a209ce4db9 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:46.503246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025c16f-9001-4fc1-8592-d0e078278307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:46.503891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025c16f-9001-4fc1-8592-d0e078278307 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:46.504173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025c16f-9001-4fc1-8592-d0e078278307 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:46.504443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025c16f-9001-4fc1-8592-d0e078278307 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:46.505069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025c16f-9001-4fc1-8592-d0e078278307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:46.507024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025c16f-9001-4fc1-8592-d0e078278307 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:46.531308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1075/2169] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:46 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:46.534985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d1850-d22f-418a-8973-5caf7646887b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:46.546706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d1850-d22f-418a-8973-5caf7646887b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:46.551550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a46a3-a228-48c4-8974-1143ea51b62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:46.552189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a46a3-a228-48c4-8974-1143ea51b62c admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:46.552467 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a46a3-a228-48c4-8974-1143ea51b62c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:46.552768 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a46a3-a228-48c4-8974-1143ea51b62c admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:46.554043 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a46a3-a228-48c4-8974-1143ea51b62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:46.555629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228a46a3-a228-48c4-8974-1143ea51b62c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:46.566819 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1095/2170] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:46.571133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025c16f-9001-4fc1-8592-d0e078278307 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:46.581063 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025c16f-9001-4fc1-8592-d0e078278307 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:46.586277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3197e4-4586-4aea-a4dd-e7fe677e3581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:46.586924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3197e4-4586-4aea-a4dd-e7fe677e3581 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:46.587206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3197e4-4586-4aea-a4dd-e7fe677e3581 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:46.587480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3197e4-4586-4aea-a4dd-e7fe677e3581 admin admin] PATH_INFO: `/v3/projects/186f31f2145548c3b1097435da6d306e/users/05f6ed5c0d6a45419f3add7085bd24c8/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:46.590558 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:46.604214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3197e4-4586-4aea-a4dd-e7fe677e3581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=186f31f2145548c3b1097435da6d306e, user_id=05f6ed5c0d6a45419f3add7085bd24c8, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:46.607147 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3197e4-4586-4aea-a4dd-e7fe677e3581 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:46.620104 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1076/2171] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:46 2022] PUT /identity/v3/projects/186f31f2145548c3b1097435da6d306e/users/05f6ed5c0d6a45419f3add7085bd24c8/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:46.624115 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a46a3-a228-48c4-8974-1143ea51b62c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:46.634035 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a46a3-a228-48c4-8974-1143ea51b62c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:46.694954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5d2d5-d4a5-4e77-87b1-e4b750dfe002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:46.695670 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5d2d5-d4a5-4e77-87b1-e4b750dfe002 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:46.696115 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5d2d5-d4a5-4e77-87b1-e4b750dfe002 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:46.696539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5d2d5-d4a5-4e77-87b1-e4b750dfe002 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:46.697665 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5d2d5-d4a5-4e77-87b1-e4b750dfe002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:46.698986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5d2d5-d4a5-4e77-87b1-e4b750dfe002 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:46.710916 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1096/2172] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:46.715079 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3197e4-4586-4aea-a4dd-e7fe677e3581 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:46.727517 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3197e4-4586-4aea-a4dd-e7fe677e3581 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:46.736005 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2186ea5-33f3-401f-ae2d-a1d0663eea3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:46.736768 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2186ea5-33f3-401f-ae2d-a1d0663eea3d admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:46.737122 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2186ea5-33f3-401f-ae2d-a1d0663eea3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:46.737469 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2186ea5-33f3-401f-ae2d-a1d0663eea3d admin admin] PATH_INFO: `/v3/projects/186f31f2145548c3b1097435da6d306e/users/05f6ed5c0d6a45419f3add7085bd24c8/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:46.739533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2186ea5-33f3-401f-ae2d-a1d0663eea3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=186f31f2145548c3b1097435da6d306e, user_id=05f6ed5c0d6a45419f3add7085bd24c8, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:46.742394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2186ea5-33f3-401f-ae2d-a1d0663eea3d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:46.755477 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1077/2173] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:46 2022] PUT /identity/v3/projects/186f31f2145548c3b1097435da6d306e/users/05f6ed5c0d6a45419f3add7085bd24c8/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:48.230757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5d2d5-d4a5-4e77-87b1-e4b750dfe002 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:48.233639 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5ab5d2d5-d4a5-4e77-87b1-e4b750dfe002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:48.264592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5d2d5-d4a5-4e77-87b1-e4b750dfe002 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:48.353326 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ebf2b-bb34-4737-a586-17b1861ed4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:48.354543 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ebf2b-bb34-4737-a586-17b1861ed4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:48.354894 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ebf2b-bb34-4737-a586-17b1861ed4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:48.355235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ebf2b-bb34-4737-a586-17b1861ed4ad admin admin] PATH_INFO: `/v3/users/35cb7bfbb282465db8b32f36f4dae016` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:48.357209 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ebf2b-bb34-4737-a586-17b1861ed4ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=35cb7bfbb282465db8b32f36f4dae016)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:48.359263 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ebf2b-bb34-4737-a586-17b1861ed4ad admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:48.408697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:48.422863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-355ebf2b-bb34-4737-a586-17b1861ed4ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35cb7bfbb282465db8b32f36f4dae016'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:48.439168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-355ebf2b-bb34-4737-a586-17b1861ed4ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35cb7bfbb282465db8b32f36f4dae016'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:48.457327 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-355ebf2b-bb34-4737-a586-17b1861ed4ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35cb7bfbb282465db8b32f36f4dae016'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:48.472216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-355ebf2b-bb34-4737-a586-17b1861ed4ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35cb7bfbb282465db8b32f36f4dae016'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:48.473674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-355ebf2b-bb34-4737-a586-17b1861ed4ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35cb7bfbb282465db8b32f36f4dae016'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:48.486764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1097/2174] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:48 2022] DELETE /identity/v3/users/35cb7bfbb282465db8b32f36f4dae016 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:48.500548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b7511-dfa9-430f-ba53-d644a85a673c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:48.501423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b7511-dfa9-430f-ba53-d644a85a673c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:48.501844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671b7511-dfa9-430f-ba53-d644a85a673c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:48.526367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-671b7511-dfa9-430f-ba53-d644a85a673c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:48.527721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-671b7511-dfa9-430f-ba53-d644a85a673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:48.601889 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1078/2175] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:48.613029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ebf2b-bb34-4737-a586-17b1861ed4ad admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:48.614289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-355ebf2b-bb34-4737-a586-17b1861ed4ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:48.687284 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ebf2b-bb34-4737-a586-17b1861ed4ad admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:48.693408 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e921e6-6966-4dab-a0a3-23c7f5aed6fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:48.694945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e921e6-6966-4dab-a0a3-23c7f5aed6fd service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:48.695230 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e921e6-6966-4dab-a0a3-23c7f5aed6fd service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:48.695735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e921e6-6966-4dab-a0a3-23c7f5aed6fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:48.709343 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e921e6-6966-4dab-a0a3-23c7f5aed6fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:48.711433 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e921e6-6966-4dab-a0a3-23c7f5aed6fd service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:48.748643 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1098/2176] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:49.228603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b7511-dfa9-430f-ba53-d644a85a673c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:49.233943 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-671b7511-dfa9-430f-ba53-d644a85a673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:49.251361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671b7511-dfa9-430f-ba53-d644a85a673c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:49.259328 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f988b0-6807-4597-9cac-caa78fdac8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:49.261771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f988b0-6807-4597-9cac-caa78fdac8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:49.262243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f988b0-6807-4597-9cac-caa78fdac8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:49.262661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f988b0-6807-4597-9cac-caa78fdac8e0 admin admin] PATH_INFO: `/v3/projects/d1b7b4531372417d9b9929a823938a7a` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:49.264160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f988b0-6807-4597-9cac-caa78fdac8e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1b7b4531372417d9b9929a823938a7a)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:49.266331 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f988b0-6807-4597-9cac-caa78fdac8e0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:49.369069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b5f988b0-6807-4597-9cac-caa78fdac8e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1b7b4531372417d9b9929a823938a7a', 'request_id': 'req-b5f988b0-6807-4597-9cac-caa78fdac8e0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:49.418532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b5f988b0-6807-4597-9cac-caa78fdac8e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1b7b4531372417d9b9929a823938a7a', 'request_id': 'req-b5f988b0-6807-4597-9cac-caa78fdac8e0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:49.477320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b5f988b0-6807-4597-9cac-caa78fdac8e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1b7b4531372417d9b9929a823938a7a', 'request_id': 'req-b5f988b0-6807-4597-9cac-caa78fdac8e0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:49.513583 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b5f988b0-6807-4597-9cac-caa78fdac8e0 admin admin] The token cache is being invalidate because project d1b7b4531372417d9b9929a823938a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:49.514171 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b5f988b0-6807-4597-9cac-caa78fdac8e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:49.519274 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b5f988b0-6807-4597-9cac-caa78fdac8e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:49.525192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1079/2177] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:49 2022] DELETE /identity/v3/projects/d1b7b4531372417d9b9929a823938a7a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:49.526510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e921e6-6966-4dab-a0a3-23c7f5aed6fd service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:49.540225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e921e6-6966-4dab-a0a3-23c7f5aed6fd service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:49.650798 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e22634-8d91-4fca-b33f-41548bac92ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:49.651477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e22634-8d91-4fca-b33f-41548bac92ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:49.651749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e22634-8d91-4fca-b33f-41548bac92ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:49.652283 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e22634-8d91-4fca-b33f-41548bac92ed admin admin] PATH_INFO: `/v3/users/92f7b76f15ac4c1f810480acce986237` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:49.654140 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e22634-8d91-4fca-b33f-41548bac92ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=92f7b76f15ac4c1f810480acce986237)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:49.656808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e22634-8d91-4fca-b33f-41548bac92ed admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:49.731941 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-41e22634-8d91-4fca-b33f-41548bac92ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92f7b76f15ac4c1f810480acce986237'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:49.744415 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-41e22634-8d91-4fca-b33f-41548bac92ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92f7b76f15ac4c1f810480acce986237'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:49.748019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:49.769562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-41e22634-8d91-4fca-b33f-41548bac92ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92f7b76f15ac4c1f810480acce986237'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:49.781519 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-41e22634-8d91-4fca-b33f-41548bac92ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92f7b76f15ac4c1f810480acce986237'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:49.784582 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-41e22634-8d91-4fca-b33f-41548bac92ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92f7b76f15ac4c1f810480acce986237'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:49.796590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1099/2178] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:49 2022] DELETE /identity/v3/users/92f7b76f15ac4c1f810480acce986237 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:51.215025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f988b0-6807-4597-9cac-caa78fdac8e0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:51.217524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b5f988b0-6807-4597-9cac-caa78fdac8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:51.247492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f988b0-6807-4597-9cac-caa78fdac8e0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:51.409692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:51.439480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ef44b-1442-44af-a2ff-f7a0841baaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:51.441113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ef44b-1442-44af-a2ff-f7a0841baaf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:51.441356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ef44b-1442-44af-a2ff-f7a0841baaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:51.441628 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ef44b-1442-44af-a2ff-f7a0841baaf0 admin admin] PATH_INFO: `/v3/projects/9eab55b5372c4eeca21d5f2e986479a9` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:51.460665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ef44b-1442-44af-a2ff-f7a0841baaf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eab55b5372c4eeca21d5f2e986479a9)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:51.469983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ef44b-1442-44af-a2ff-f7a0841baaf0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:51.594273 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e45ef44b-1442-44af-a2ff-f7a0841baaf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eab55b5372c4eeca21d5f2e986479a9', 'request_id': 'req-e45ef44b-1442-44af-a2ff-f7a0841baaf0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:51.609931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e45ef44b-1442-44af-a2ff-f7a0841baaf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eab55b5372c4eeca21d5f2e986479a9', 'request_id': 'req-e45ef44b-1442-44af-a2ff-f7a0841baaf0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:51.638542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e45ef44b-1442-44af-a2ff-f7a0841baaf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eab55b5372c4eeca21d5f2e986479a9', 'request_id': 'req-e45ef44b-1442-44af-a2ff-f7a0841baaf0'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:51.673426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e45ef44b-1442-44af-a2ff-f7a0841baaf0 admin admin] The token cache is being invalidate because project 9eab55b5372c4eeca21d5f2e986479a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:51.673773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e45ef44b-1442-44af-a2ff-f7a0841baaf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:51.676164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e45ef44b-1442-44af-a2ff-f7a0841baaf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:51.681800 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1080/2179] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:51 2022] DELETE /identity/v3/projects/9eab55b5372c4eeca21d5f2e986479a9 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:51.686846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97818fdf-cc23-454a-9d09-f5ee34654fb7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:51.687874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97818fdf-cc23-454a-9d09-f5ee34654fb7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:51.688713 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97818fdf-cc23-454a-9d09-f5ee34654fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:51.752777 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-97818fdf-cc23-454a-9d09-f5ee34654fb7 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:51.758218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-97818fdf-cc23-454a-9d09-f5ee34654fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:51.906539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1100/2180] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:51.908472 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548d001-155e-4ff8-99af-1271e34ec2f9 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:51.908794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548d001-155e-4ff8-99af-1271e34ec2f9 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:51.909045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548d001-155e-4ff8-99af-1271e34ec2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:51.948643 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9548d001-155e-4ff8-99af-1271e34ec2f9 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:51.950188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9548d001-155e-4ff8-99af-1271e34ec2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:51.975990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1081/2181] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:51.981734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97818fdf-cc23-454a-9d09-f5ee34654fb7 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:51.999215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97818fdf-cc23-454a-9d09-f5ee34654fb7 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:52.007099 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fddd49-8010-4c6f-8fc3-a1830855937d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:52.007750 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fddd49-8010-4c6f-8fc3-a1830855937d admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:52.008212 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fddd49-8010-4c6f-8fc3-a1830855937d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:52.008492 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fddd49-8010-4c6f-8fc3-a1830855937d admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:52.009483 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fddd49-8010-4c6f-8fc3-a1830855937d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:52.013911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fddd49-8010-4c6f-8fc3-a1830855937d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:52.016339 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fddd49-8010-4c6f-8fc3-a1830855937d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:52.035756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1101/2182] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:53:52.039796 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9548d001-155e-4ff8-99af-1271e34ec2f9 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:52.061311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9548d001-155e-4ff8-99af-1271e34ec2f9 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:52.099560 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421703c-6de4-4f79-bacb-de7e26a7384a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:52.100283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421703c-6de4-4f79-bacb-de7e26a7384a admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:52.100571 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421703c-6de4-4f79-bacb-de7e26a7384a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:52.100845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421703c-6de4-4f79-bacb-de7e26a7384a admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:52.101534 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421703c-6de4-4f79-bacb-de7e26a7384a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:52.103442 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421703c-6de4-4f79-bacb-de7e26a7384a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:52.176443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1082/2183] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:52 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:53:52.176954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fddd49-8010-4c6f-8fc3-a1830855937d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:52.193531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fddd49-8010-4c6f-8fc3-a1830855937d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:52.341373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a0f29-7479-4b85-8b83-87d3098c547f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:52.342011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a0f29-7479-4b85-8b83-87d3098c547f admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:52.342335 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a0f29-7479-4b85-8b83-87d3098c547f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:52.342611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a0f29-7479-4b85-8b83-87d3098c547f admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:52.343254 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a0f29-7479-4b85-8b83-87d3098c547f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:52.345760 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a0f29-7479-4b85-8b83-87d3098c547f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:52.381218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421703c-6de4-4f79-bacb-de7e26a7384a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:52.393636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1102/2184] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:52 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:52.404185 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421703c-6de4-4f79-bacb-de7e26a7384a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:52.410003 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d599aa-b78a-4546-a2a7-d39c569698c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:52.410797 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d599aa-b78a-4546-a2a7-d39c569698c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:52.411183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d599aa-b78a-4546-a2a7-d39c569698c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:52.411575 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d599aa-b78a-4546-a2a7-d39c569698c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:52.413600 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d599aa-b78a-4546-a2a7-d39c569698c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:52.416826 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d599aa-b78a-4546-a2a7-d39c569698c8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:52.448776 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1083/2185] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:52.452602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a0f29-7479-4b85-8b83-87d3098c547f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:52.465566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a0f29-7479-4b85-8b83-87d3098c547f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:52.476411 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03999ca0-3bb4-43dd-9fdf-b05add6552bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:52.477295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03999ca0-3bb4-43dd-9fdf-b05add6552bd admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:52.477585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03999ca0-3bb4-43dd-9fdf-b05add6552bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:52.478092 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03999ca0-3bb4-43dd-9fdf-b05add6552bd admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:52.478765 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03999ca0-3bb4-43dd-9fdf-b05add6552bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:52.480578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03999ca0-3bb4-43dd-9fdf-b05add6552bd admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:52.511590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1103/2186] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:53:52.516119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d599aa-b78a-4546-a2a7-d39c569698c8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:52.529283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d599aa-b78a-4546-a2a7-d39c569698c8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:52.537101 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e542fe4-6bbe-4da3-8a5f-129f6198f8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:52.538370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e542fe4-6bbe-4da3-8a5f-129f6198f8a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:52.538663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e542fe4-6bbe-4da3-8a5f-129f6198f8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:52.538937 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e542fe4-6bbe-4da3-8a5f-129f6198f8a4 admin admin] PATH_INFO: `/v3/projects/0d6791fca5494cd5beb1a1505f285b14/users/fe1f690c45c641d58ff5a9c45f60722f/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:52.571254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e542fe4-6bbe-4da3-8a5f-129f6198f8a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6791fca5494cd5beb1a1505f285b14, user_id=fe1f690c45c641d58ff5a9c45f60722f, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:52.576938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e542fe4-6bbe-4da3-8a5f-129f6198f8a4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:52.615432 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1084/2187] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:52 2022] PUT /identity/v3/projects/0d6791fca5494cd5beb1a1505f285b14/users/fe1f690c45c641d58ff5a9c45f60722f/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:52.625767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d848784-4cc2-4361-b3e4-878b8930d504 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:52.626530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d848784-4cc2-4361-b3e4-878b8930d504 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:52.634648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d848784-4cc2-4361-b3e4-878b8930d504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:52.743554 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5d848784-4cc2-4361-b3e4-878b8930d504 None None] MFA Rules not processed for user `fe1f690c45c641d58ff5a9c45f60722f`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:52.745352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5d848784-4cc2-4361-b3e4-878b8930d504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:52.832950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:52.895521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1104/2188] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:53:52.923941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e542fe4-6bbe-4da3-8a5f-129f6198f8a4 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:52.926054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0e542fe4-6bbe-4da3-8a5f-129f6198f8a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:53.040982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e542fe4-6bbe-4da3-8a5f-129f6198f8a4 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:53.046746 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c63c30-d18d-4c62-86ce-795058b1a837 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:53.047716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c63c30-d18d-4c62-86ce-795058b1a837 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:53.048154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c63c30-d18d-4c62-86ce-795058b1a837 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:53.048532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c63c30-d18d-4c62-86ce-795058b1a837 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:53.078853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c63c30-d18d-4c62-86ce-795058b1a837 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:53.080650 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c63c30-d18d-4c62-86ce-795058b1a837 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:53.108705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1085/2189] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:53.144818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d848784-4cc2-4361-b3e4-878b8930d504 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:53.315102 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d848784-4cc2-4361-b3e4-878b8930d504 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:53.328820 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a27d5-7a1e-4259-8887-cb418880efe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:53.329494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a27d5-7a1e-4259-8887-cb418880efe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:53.329777 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a27d5-7a1e-4259-8887-cb418880efe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:53.330364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a27d5-7a1e-4259-8887-cb418880efe5 admin admin] PATH_INFO: `/v3/users/fe1f690c45c641d58ff5a9c45f60722f` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:53.333675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a27d5-7a1e-4259-8887-cb418880efe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe1f690c45c641d58ff5a9c45f60722f)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:53.340352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a27d5-7a1e-4259-8887-cb418880efe5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:53.436860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9c4a27d5-7a1e-4259-8887-cb418880efe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe1f690c45c641d58ff5a9c45f60722f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:53.447628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9c4a27d5-7a1e-4259-8887-cb418880efe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe1f690c45c641d58ff5a9c45f60722f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:53.468508 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9c4a27d5-7a1e-4259-8887-cb418880efe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe1f690c45c641d58ff5a9c45f60722f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:53.488992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9c4a27d5-7a1e-4259-8887-cb418880efe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe1f690c45c641d58ff5a9c45f60722f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:53.489648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9c4a27d5-7a1e-4259-8887-cb418880efe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe1f690c45c641d58ff5a9c45f60722f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:53.510055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1105/2190] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:53 2022] DELETE /identity/v3/users/fe1f690c45c641d58ff5a9c45f60722f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:53.514404 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273a233-5a04-4582-8a7e-cc3930834261 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:53.514919 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273a233-5a04-4582-8a7e-cc3930834261 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:53.515023 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273a233-5a04-4582-8a7e-cc3930834261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:53.551307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b273a233-5a04-4582-8a7e-cc3930834261 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:53.552837 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b273a233-5a04-4582-8a7e-cc3930834261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:53.595982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:53.663708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81a403-1057-4181-b9a3-03a5fe8572bd None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:53.664041 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81a403-1057-4181-b9a3-03a5fe8572bd None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:53.664319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81a403-1057-4181-b9a3-03a5fe8572bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:53.698493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1086/2191] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:53.709859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273a233-5a04-4582-8a7e-cc3930834261 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:53.711328 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b273a233-5a04-4582-8a7e-cc3930834261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:53.722685 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3f81a403-1057-4181-b9a3-03a5fe8572bd None None] MFA Rules not processed for user `05f6ed5c0d6a45419f3add7085bd24c8`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:53.723991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3f81a403-1057-4181-b9a3-03a5fe8572bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:53.791942 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273a233-5a04-4582-8a7e-cc3930834261 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:53.795668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1c470-ea45-4376-88d0-f7e91f6d8a28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:53.796326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1c470-ea45-4376-88d0-f7e91f6d8a28 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:53.796610 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1c470-ea45-4376-88d0-f7e91f6d8a28 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:53.796883 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1c470-ea45-4376-88d0-f7e91f6d8a28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:53.815780 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:53.825226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1c470-ea45-4376-88d0-f7e91f6d8a28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:53.829991 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1c470-ea45-4376-88d0-f7e91f6d8a28 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:53.844773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1087/2192] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:53.872790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1106/2193] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:53.893244 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1c470-ea45-4376-88d0-f7e91f6d8a28 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:53.894763 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5fb1c470-ea45-4376-88d0-f7e91f6d8a28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:54.050857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1c470-ea45-4376-88d0-f7e91f6d8a28 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:54.061372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f971881-e4b5-4ed2-a6a0-a02d8b4b4a5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:54.062725 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f971881-e4b5-4ed2-a6a0-a02d8b4b4a5f service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:54.063222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f971881-e4b5-4ed2-a6a0-a02d8b4b4a5f service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:54.063517 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f971881-e4b5-4ed2-a6a0-a02d8b4b4a5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:54.066861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7f971881-e4b5-4ed2-a6a0-a02d8b4b4a5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:54.155614 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f971881-e4b5-4ed2-a6a0-a02d8b4b4a5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:54.162126 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f971881-e4b5-4ed2-a6a0-a02d8b4b4a5f service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:54.173763 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1088/2194] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:53:54.257642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81a403-1057-4181-b9a3-03a5fe8572bd None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:54.259541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3f81a403-1057-4181-b9a3-03a5fe8572bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:54.280896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81a403-1057-4181-b9a3-03a5fe8572bd None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:54.301008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8429be-8b56-4471-9578-bae3188cb3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:54.303309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8429be-8b56-4471-9578-bae3188cb3be admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:54.303769 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8429be-8b56-4471-9578-bae3188cb3be admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:54.304289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8429be-8b56-4471-9578-bae3188cb3be admin admin] PATH_INFO: `/v3/projects/0d6791fca5494cd5beb1a1505f285b14` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:54.307603 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8429be-8b56-4471-9578-bae3188cb3be admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d6791fca5494cd5beb1a1505f285b14)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:54.311034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8429be-8b56-4471-9578-bae3188cb3be admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:54.397687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5d8429be-8b56-4471-9578-bae3188cb3be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d6791fca5494cd5beb1a1505f285b14', 'request_id': 'req-5d8429be-8b56-4471-9578-bae3188cb3be'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:54.412864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5d8429be-8b56-4471-9578-bae3188cb3be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d6791fca5494cd5beb1a1505f285b14', 'request_id': 'req-5d8429be-8b56-4471-9578-bae3188cb3be'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:54.420647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5d8429be-8b56-4471-9578-bae3188cb3be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d6791fca5494cd5beb1a1505f285b14', 'request_id': 'req-5d8429be-8b56-4471-9578-bae3188cb3be'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:54.437651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5d8429be-8b56-4471-9578-bae3188cb3be admin admin] The token cache is being invalidate because project 0d6791fca5494cd5beb1a1505f285b14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:54.437979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5d8429be-8b56-4471-9578-bae3188cb3be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:54.438676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5d8429be-8b56-4471-9578-bae3188cb3be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:54.441263 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1107/2195] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:54 2022] DELETE /identity/v3/projects/0d6791fca5494cd5beb1a1505f285b14 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:55.825419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedc080-dbb3-4cf3-b2a3-b4b1cf9fd9d8 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:55.826418 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedc080-dbb3-4cf3-b2a3-b4b1cf9fd9d8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:55.827535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedc080-dbb3-4cf3-b2a3-b4b1cf9fd9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:55.842344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:55.876028 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1bedc080-dbb3-4cf3-b2a3-b4b1cf9fd9d8 None None] MFA Rules not processed for user `218c699b6ddd43a4bcf9b842c59445f5`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:55.878975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1bedc080-dbb3-4cf3-b2a3-b4b1cf9fd9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:56.010334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1089/2196] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:56.230680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8429be-8b56-4471-9578-bae3188cb3be admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:56.233260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5d8429be-8b56-4471-9578-bae3188cb3be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:56.357680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8429be-8b56-4471-9578-bae3188cb3be admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:56.362852 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27471a01-9abb-444b-9c55-b067769057a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:56.363441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27471a01-9abb-444b-9c55-b067769057a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:56.363728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27471a01-9abb-444b-9c55-b067769057a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:56.364171 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27471a01-9abb-444b-9c55-b067769057a9 admin admin] PATH_INFO: `/v3/users/49b775647ffb4500911c8ec8cf2ec804` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:56.365844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27471a01-9abb-444b-9c55-b067769057a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b775647ffb4500911c8ec8cf2ec804)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:56.367873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27471a01-9abb-444b-9c55-b067769057a9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:56.491428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-27471a01-9abb-444b-9c55-b067769057a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b775647ffb4500911c8ec8cf2ec804'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:56.505626 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-27471a01-9abb-444b-9c55-b067769057a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b775647ffb4500911c8ec8cf2ec804'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:56.513098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:56.528233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-27471a01-9abb-444b-9c55-b067769057a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b775647ffb4500911c8ec8cf2ec804'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:56.541907 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-27471a01-9abb-444b-9c55-b067769057a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b775647ffb4500911c8ec8cf2ec804'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:56.542657 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-27471a01-9abb-444b-9c55-b067769057a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b775647ffb4500911c8ec8cf2ec804'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:56.552561 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1108/2197] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:56 2022] DELETE /identity/v3/users/49b775647ffb4500911c8ec8cf2ec804 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:56.685245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedc080-dbb3-4cf3-b2a3-b4b1cf9fd9d8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:56.687635 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1bedc080-dbb3-4cf3-b2a3-b4b1cf9fd9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:56.715864 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedc080-dbb3-4cf3-b2a3-b4b1cf9fd9d8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:56.822889 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beabf9a-d144-4647-b3a6-4edc61b9a6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:56.823533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beabf9a-d144-4647-b3a6-4edc61b9a6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:56.823813 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beabf9a-d144-4647-b3a6-4edc61b9a6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:56.824137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beabf9a-d144-4647-b3a6-4edc61b9a6ac admin admin] PATH_INFO: `/v3/users/88d1c9c7ec5e409698fdf7bad80210c5` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:56.827032 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beabf9a-d144-4647-b3a6-4edc61b9a6ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d1c9c7ec5e409698fdf7bad80210c5)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:56.852881 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beabf9a-d144-4647-b3a6-4edc61b9a6ac admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:56.868349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27471a01-9abb-444b-9c55-b067769057a9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:56.870011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-27471a01-9abb-444b-9c55-b067769057a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:56.900057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27471a01-9abb-444b-9c55-b067769057a9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:56.923180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb301d17-a45f-412a-a74e-abea0516854f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:56.924071 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb301d17-a45f-412a-a74e-abea0516854f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:56.924379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb301d17-a45f-412a-a74e-abea0516854f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:56.924686 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb301d17-a45f-412a-a74e-abea0516854f admin admin] PATH_INFO: `/v3/projects/a0009b7d4f2545ffab964938384d40b6` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:56.926660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb301d17-a45f-412a-a74e-abea0516854f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0009b7d4f2545ffab964938384d40b6)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:56.936646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb301d17-a45f-412a-a74e-abea0516854f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:56.967212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3beabf9a-d144-4647-b3a6-4edc61b9a6ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d1c9c7ec5e409698fdf7bad80210c5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:56.977541 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3beabf9a-d144-4647-b3a6-4edc61b9a6ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d1c9c7ec5e409698fdf7bad80210c5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:56.999901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3beabf9a-d144-4647-b3a6-4edc61b9a6ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d1c9c7ec5e409698fdf7bad80210c5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:57.017306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3beabf9a-d144-4647-b3a6-4edc61b9a6ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d1c9c7ec5e409698fdf7bad80210c5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:57.018209 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-3beabf9a-d144-4647-b3a6-4edc61b9a6ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d1c9c7ec5e409698fdf7bad80210c5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:57.028472 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1090/2198] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:53:56 2022] DELETE /identity/v3/users/88d1c9c7ec5e409698fdf7bad80210c5 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:57.033525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a5c37-c5c9-49e0-8077-9c6d13dfedfd None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:57.034116 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a5c37-c5c9-49e0-8077-9c6d13dfedfd None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:57.034766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a5c37-c5c9-49e0-8077-9c6d13dfedfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:57.069803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3c0a5c37-c5c9-49e0-8077-9c6d13dfedfd None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:57.074051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3c0a5c37-c5c9-49e0-8077-9c6d13dfedfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:57.081923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fb301d17-a45f-412a-a74e-abea0516854f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0009b7d4f2545ffab964938384d40b6', 'request_id': 'req-fb301d17-a45f-412a-a74e-abea0516854f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:57.093938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fb301d17-a45f-412a-a74e-abea0516854f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0009b7d4f2545ffab964938384d40b6', 'request_id': 'req-fb301d17-a45f-412a-a74e-abea0516854f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:57.104107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fb301d17-a45f-412a-a74e-abea0516854f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0009b7d4f2545ffab964938384d40b6', 'request_id': 'req-fb301d17-a45f-412a-a74e-abea0516854f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:57.143605 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fb301d17-a45f-412a-a74e-abea0516854f admin admin] The token cache is being invalidate because project a0009b7d4f2545ffab964938384d40b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:57.143994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fb301d17-a45f-412a-a74e-abea0516854f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:57.145750 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fb301d17-a45f-412a-a74e-abea0516854f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:57.148161 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1109/2199] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:56 2022] DELETE /identity/v3/projects/a0009b7d4f2545ffab964938384d40b6 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:57.153356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70bd48-8cf7-45bf-b17c-2a201f4b4a28 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:57.153647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70bd48-8cf7-45bf-b17c-2a201f4b4a28 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:57.153929 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d70bd48-8cf7-45bf-b17c-2a201f4b4a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:57.156401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:57.194082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4d70bd48-8cf7-45bf-b17c-2a201f4b4a28 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:57.195270 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4d70bd48-8cf7-45bf-b17c-2a201f4b4a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:57.218102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1091/2200] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:57.231174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a5c37-c5c9-49e0-8077-9c6d13dfedfd None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:57.242708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3c0a5c37-c5c9-49e0-8077-9c6d13dfedfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:57.283112 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1110/2201] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:53:57.291994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd19be3-e598-4920-8cb0-0a33570a2e2b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:57.292281 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd19be3-e598-4920-8cb0-0a33570a2e2b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:57.292629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd19be3-e598-4920-8cb0-0a33570a2e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:57.316584 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2dd19be3-e598-4920-8cb0-0a33570a2e2b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:57.317816 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2dd19be3-e598-4920-8cb0-0a33570a2e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:57.324256 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a5c37-c5c9-49e0-8077-9c6d13dfedfd None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:57.328167 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1111/2202] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:57.328295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d663bf0-caf8-4928-b729-5d51f9573e5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:57.329040 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d663bf0-caf8-4928-b729-5d51f9573e5d service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:57.329418 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d663bf0-caf8-4928-b729-5d51f9573e5d service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:57.329760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d663bf0-caf8-4928-b729-5d51f9573e5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:57.332385 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd19be3-e598-4920-8cb0-0a33570a2e2b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:57.341332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d663bf0-caf8-4928-b729-5d51f9573e5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:57.342682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd19be3-e598-4920-8cb0-0a33570a2e2b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:57.342931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d663bf0-caf8-4928-b729-5d51f9573e5d service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:57.347953 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb80222-6cc7-4d8d-86b8-d26991dc5c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:57.349160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb80222-6cc7-4d8d-86b8-d26991dc5c3f admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:57.349160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb80222-6cc7-4d8d-86b8-d26991dc5c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:57.349160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb80222-6cc7-4d8d-86b8-d26991dc5c3f admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:57.350309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb80222-6cc7-4d8d-86b8-d26991dc5c3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:57.350615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb80222-6cc7-4d8d-86b8-d26991dc5c3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:57.352243 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb80222-6cc7-4d8d-86b8-d26991dc5c3f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:57.356583 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1092/2203] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:53:57.368026 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1112/2204] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:53:57.370051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d663bf0-caf8-4928-b729-5d51f9573e5d service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:57.381339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d663bf0-caf8-4928-b729-5d51f9573e5d service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:57.386539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d870f49-d2d3-4cb5-8ed8-43b93018bbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:57.387166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d870f49-d2d3-4cb5-8ed8-43b93018bbca admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:57.387451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d870f49-d2d3-4cb5-8ed8-43b93018bbca admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:57.387722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d870f49-d2d3-4cb5-8ed8-43b93018bbca admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:57.388512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d870f49-d2d3-4cb5-8ed8-43b93018bbca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:57.390473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d870f49-d2d3-4cb5-8ed8-43b93018bbca admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:57.434337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1093/2205] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:57 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:57.434728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb80222-6cc7-4d8d-86b8-d26991dc5c3f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:57.449270 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb80222-6cc7-4d8d-86b8-d26991dc5c3f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:57.504623 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:57.554236 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fe225-840e-4787-8b9d-7563214cc299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:57.555503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fe225-840e-4787-8b9d-7563214cc299 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:57.556173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fe225-840e-4787-8b9d-7563214cc299 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:57.556790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fe225-840e-4787-8b9d-7563214cc299 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:57.557428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9fe225-840e-4787-8b9d-7563214cc299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:57.559530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9fe225-840e-4787-8b9d-7563214cc299 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:57.586769 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1113/2206] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:57 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:57.591097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d870f49-d2d3-4cb5-8ed8-43b93018bbca admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:57.602373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d870f49-d2d3-4cb5-8ed8-43b93018bbca admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:57.608869 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7194e0d-c246-4aad-a455-a3b8bc6b68fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:57.609487 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7194e0d-c246-4aad-a455-a3b8bc6b68fe admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:57.609759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7194e0d-c246-4aad-a455-a3b8bc6b68fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:57.610054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7194e0d-c246-4aad-a455-a3b8bc6b68fe admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:57.610846 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7194e0d-c246-4aad-a455-a3b8bc6b68fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:57.612699 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7194e0d-c246-4aad-a455-a3b8bc6b68fe admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:57.625933 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1094/2207] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:57.630665 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fe225-840e-4787-8b9d-7563214cc299 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:57.642832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fe225-840e-4787-8b9d-7563214cc299 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:57.648438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6b8c7-4b27-46d6-bdaa-b5a1f4bba381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:57.649235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6b8c7-4b27-46d6-bdaa-b5a1f4bba381 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:57.649522 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6b8c7-4b27-46d6-bdaa-b5a1f4bba381 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:57.651433 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6b8c7-4b27-46d6-bdaa-b5a1f4bba381 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:57.652423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6b8c7-4b27-46d6-bdaa-b5a1f4bba381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:57.654113 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6b8c7-4b27-46d6-bdaa-b5a1f4bba381 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:57.671332 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1114/2208] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:53:57.673070 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7194e0d-c246-4aad-a455-a3b8bc6b68fe admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:57.695761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7194e0d-c246-4aad-a455-a3b8bc6b68fe admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:57.702844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fb951-05c1-4b25-b354-0cc942e2f1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:57.703530 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fb951-05c1-4b25-b354-0cc942e2f1ec admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:57.703816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fb951-05c1-4b25-b354-0cc942e2f1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:57.704236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15fb951-05c1-4b25-b354-0cc942e2f1ec admin admin] PATH_INFO: `/v3/projects/9a431be4c2f247e2955883eab6a5f555/users/73ef9d62f31d4135b6f079a26d791ca5/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:57.729848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fb951-05c1-4b25-b354-0cc942e2f1ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a431be4c2f247e2955883eab6a5f555, user_id=73ef9d62f31d4135b6f079a26d791ca5, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:57.733085 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15fb951-05c1-4b25-b354-0cc942e2f1ec admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:57.750282 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1095/2209] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:57 2022] PUT /identity/v3/projects/9a431be4c2f247e2955883eab6a5f555/users/73ef9d62f31d4135b6f079a26d791ca5/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:57.755479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa87c0-5587-477c-b9bc-14d4dd94a1dc None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:57.755886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa87c0-5587-477c-b9bc-14d4dd94a1dc None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:57.756159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa87c0-5587-477c-b9bc-14d4dd94a1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:57.768952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fb951-05c1-4b25-b354-0cc942e2f1ec admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:57.771410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f15fb951-05c1-4b25-b354-0cc942e2f1ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:57.797902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2daa87c0-5587-477c-b9bc-14d4dd94a1dc None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:57.799197 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2daa87c0-5587-477c-b9bc-14d4dd94a1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:57.862443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15fb951-05c1-4b25-b354-0cc942e2f1ec admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:57.869704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391f026-d360-409d-a2e9-63b66b0daf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:57.870349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391f026-d360-409d-a2e9-63b66b0daf73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:57.870634 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391f026-d360-409d-a2e9-63b66b0daf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:57.870932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391f026-d360-409d-a2e9-63b66b0daf73 admin admin] PATH_INFO: `/v3/projects/33e2ac6867e748d4af2042a95b5d0b58` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:57.872139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391f026-d360-409d-a2e9-63b66b0daf73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33e2ac6867e748d4af2042a95b5d0b58)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:57.874078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391f026-d360-409d-a2e9-63b66b0daf73 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:57.879876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1115/2210] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:57.888411 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa87c0-5587-477c-b9bc-14d4dd94a1dc None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:57.895252 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:57.934288 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1391f026-d360-409d-a2e9-63b66b0daf73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33e2ac6867e748d4af2042a95b5d0b58', 'request_id': 'req-1391f026-d360-409d-a2e9-63b66b0daf73'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:57.942092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1391f026-d360-409d-a2e9-63b66b0daf73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33e2ac6867e748d4af2042a95b5d0b58', 'request_id': 'req-1391f026-d360-409d-a2e9-63b66b0daf73'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:57.951007 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1391f026-d360-409d-a2e9-63b66b0daf73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33e2ac6867e748d4af2042a95b5d0b58', 'request_id': 'req-1391f026-d360-409d-a2e9-63b66b0daf73'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:57.962674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa87c0-5587-477c-b9bc-14d4dd94a1dc None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:57.964908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1391f026-d360-409d-a2e9-63b66b0daf73 admin admin] The token cache is being invalidate because project 33e2ac6867e748d4af2042a95b5d0b58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:53:57.965213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1391f026-d360-409d-a2e9-63b66b0daf73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:57.965929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1391f026-d360-409d-a2e9-63b66b0daf73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:53:57.967725 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1096/2211] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:53:57 2022] DELETE /identity/v3/projects/33e2ac6867e748d4af2042a95b5d0b58 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:57.972801 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2e257-6588-4a2e-92ab-e85333e36593 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:57.973088 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2e257-6588-4a2e-92ab-e85333e36593 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:57.973854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2e257-6588-4a2e-92ab-e85333e36593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:57.997636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f4b2e257-6588-4a2e-92ab-e85333e36593 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:57.998958 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f4b2e257-6588-4a2e-92ab-e85333e36593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:58.025556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c2e2b-50cc-4f3d-90d2-272affbff9b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:58.026181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c2e2b-50cc-4f3d-90d2-272affbff9b2 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:58.026472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c2e2b-50cc-4f3d-90d2-272affbff9b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:58.026755 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c2e2b-50cc-4f3d-90d2-272affbff9b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:58.029105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-711c2e2b-50cc-4f3d-90d2-272affbff9b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:58.074807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:58.076879 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1097/2212] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:58.086351 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124af823-b9fa-4cf4-9066-dc6a1f421422 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:58.086723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124af823-b9fa-4cf4-9066-dc6a1f421422 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:58.087105 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124af823-b9fa-4cf4-9066-dc6a1f421422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:58.108128 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-124af823-b9fa-4cf4-9066-dc6a1f421422 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:58.109337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-124af823-b9fa-4cf4-9066-dc6a1f421422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:58.116802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c2e2b-50cc-4f3d-90d2-272affbff9b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:58.118227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1098/2213] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:58.118365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c2e2b-50cc-4f3d-90d2-272affbff9b2 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:58.122528 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124af823-b9fa-4cf4-9066-dc6a1f421422 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:58.129174 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1116/2214] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:53:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:53:58.133864 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124af823-b9fa-4cf4-9066-dc6a1f421422 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:58.138964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68b3cb-aac5-4306-973f-346ea3f68980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:58.139419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68b3cb-aac5-4306-973f-346ea3f68980 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:58.139950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68b3cb-aac5-4306-973f-346ea3f68980 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:58.140232 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68b3cb-aac5-4306-973f-346ea3f68980 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:58.141054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68b3cb-aac5-4306-973f-346ea3f68980 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:53:58.141370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68b3cb-aac5-4306-973f-346ea3f68980 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:58.142995 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68b3cb-aac5-4306-973f-346ea3f68980 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:58.157061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1099/2215] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 14:53:58.161771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c2e2b-50cc-4f3d-90d2-272affbff9b2 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:58.172538 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c2e2b-50cc-4f3d-90d2-272affbff9b2 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:58.177749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1acfde-55a5-492e-9958-325d72422d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:58.178372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1acfde-55a5-492e-9958-325d72422d3d admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:58.178658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1acfde-55a5-492e-9958-325d72422d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:58.178933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1acfde-55a5-492e-9958-325d72422d3d admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:58.179626 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1acfde-55a5-492e-9958-325d72422d3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:58.181954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1acfde-55a5-492e-9958-325d72422d3d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:58.216218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1117/2216] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:58 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:58.217736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68b3cb-aac5-4306-973f-346ea3f68980 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:58.230208 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68b3cb-aac5-4306-973f-346ea3f68980 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:58.298338 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f1bf3-dfbb-4b90-b61d-f0387e0cd55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:58.299020 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f1bf3-dfbb-4b90-b61d-f0387e0cd55c admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:58.299276 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f1bf3-dfbb-4b90-b61d-f0387e0cd55c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:58.299537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f1bf3-dfbb-4b90-b61d-f0387e0cd55c admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:58.300315 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f1bf3-dfbb-4b90-b61d-f0387e0cd55c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:58.302259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f1bf3-dfbb-4b90-b61d-f0387e0cd55c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:58.329295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1100/2217] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:58 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:58.329846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1acfde-55a5-492e-9958-325d72422d3d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:58.340873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1acfde-55a5-492e-9958-325d72422d3d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:58.346364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55754c2b-eddf-4954-9ff4-f94550bbdbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:58.347153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55754c2b-eddf-4954-9ff4-f94550bbdbf3 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:58.347441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55754c2b-eddf-4954-9ff4-f94550bbdbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:58.347739 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55754c2b-eddf-4954-9ff4-f94550bbdbf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:58.348404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55754c2b-eddf-4954-9ff4-f94550bbdbf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:58.350039 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55754c2b-eddf-4954-9ff4-f94550bbdbf3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:58.363098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1118/2218] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:53:58.367352 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f1bf3-dfbb-4b90-b61d-f0387e0cd55c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:58.378019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f1bf3-dfbb-4b90-b61d-f0387e0cd55c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:58.383257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c51777-5d15-4064-be4d-3147a518461b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:58.383886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c51777-5d15-4064-be4d-3147a518461b admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:58.384170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c51777-5d15-4064-be4d-3147a518461b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:58.384473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c51777-5d15-4064-be4d-3147a518461b admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:58.385108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c51777-5d15-4064-be4d-3147a518461b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:58.386757 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c51777-5d15-4064-be4d-3147a518461b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:58.399517 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1101/2219] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:58.403512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55754c2b-eddf-4954-9ff4-f94550bbdbf3 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:58.413781 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55754c2b-eddf-4954-9ff4-f94550bbdbf3 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:58.418568 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6d75d-2705-4200-8ef8-7fa0d097f4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:58.419235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6d75d-2705-4200-8ef8-7fa0d097f4e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:58.419533 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6d75d-2705-4200-8ef8-7fa0d097f4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:58.420088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6d75d-2705-4200-8ef8-7fa0d097f4e5 admin admin] PATH_INFO: `/v3/projects/1656aecf459c46919e0aee846f064586/users/017efbc4c7684f48a9893b12f6fd19d4/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:58.436250 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6d75d-2705-4200-8ef8-7fa0d097f4e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1656aecf459c46919e0aee846f064586, user_id=017efbc4c7684f48a9893b12f6fd19d4, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:58.438924 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6d75d-2705-4200-8ef8-7fa0d097f4e5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:58.462248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1119/2220] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:58 2022] PUT /identity/v3/projects/1656aecf459c46919e0aee846f064586/users/017efbc4c7684f48a9893b12f6fd19d4/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:53:58.466058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac6b1d-7bfd-4c4e-8d45-a4b9a1c6d3b9 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:58.466494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac6b1d-7bfd-4c4e-8d45-a4b9a1c6d3b9 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:58.466775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac6b1d-7bfd-4c4e-8d45-a4b9a1c6d3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:58.493147 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:58.525526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2fac6b1d-7bfd-4c4e-8d45-a4b9a1c6d3b9 None None] MFA Rules not processed for user `017efbc4c7684f48a9893b12f6fd19d4`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:58.526798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2fac6b1d-7bfd-4c4e-8d45-a4b9a1c6d3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:58.620014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1102/2221] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:58.624739 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6d75d-2705-4200-8ef8-7fa0d097f4e5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:58.696743 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6d75d-2705-4200-8ef8-7fa0d097f4e5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:58.704100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073e145-3106-4da7-bde6-308ef5184dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:58.704791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073e145-3106-4da7-bde6-308ef5184dea admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:58.705250 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073e145-3106-4da7-bde6-308ef5184dea admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:58.705394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073e145-3106-4da7-bde6-308ef5184dea admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:58.706021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1073e145-3106-4da7-bde6-308ef5184dea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:58.708149 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1073e145-3106-4da7-bde6-308ef5184dea admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:58.758523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1120/2222] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:58 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:58.759629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac6b1d-7bfd-4c4e-8d45-a4b9a1c6d3b9 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:58.781439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac6b1d-7bfd-4c4e-8d45-a4b9a1c6d3b9 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:58.904094 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47d889-0549-4546-be89-b3191430e58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:58.904928 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47d889-0549-4546-be89-b3191430e58c admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:58.905319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47d889-0549-4546-be89-b3191430e58c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:58.905694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47d889-0549-4546-be89-b3191430e58c admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:58.906701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47d889-0549-4546-be89-b3191430e58c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:58.908789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47d889-0549-4546-be89-b3191430e58c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:58.941211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1103/2223] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:58 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:58.945274 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073e145-3106-4da7-bde6-308ef5184dea admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:58.968690 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073e145-3106-4da7-bde6-308ef5184dea admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:58.978241 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a2dfa-776a-4abe-bd8f-91b64d583d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:58.978936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a2dfa-776a-4abe-bd8f-91b64d583d26 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:58.979168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a2dfa-776a-4abe-bd8f-91b64d583d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:58.979451 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a2dfa-776a-4abe-bd8f-91b64d583d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:58.980309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a2dfa-776a-4abe-bd8f-91b64d583d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:58.981950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a2dfa-776a-4abe-bd8f-91b64d583d26 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:58.998734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1121/2224] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:59.002184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47d889-0549-4546-be89-b3191430e58c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:59.015596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47d889-0549-4546-be89-b3191430e58c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:59.022427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab7ec5-2baa-4b61-b591-41799142cf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:59.023624 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab7ec5-2baa-4b61-b591-41799142cf1e admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:59.024122 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab7ec5-2baa-4b61-b591-41799142cf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:59.024644 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab7ec5-2baa-4b61-b591-41799142cf1e admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:59.026288 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab7ec5-2baa-4b61-b591-41799142cf1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:59.028119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab7ec5-2baa-4b61-b591-41799142cf1e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:59.041304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1104/2225] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:59.045363 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a2dfa-776a-4abe-bd8f-91b64d583d26 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:59.057470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a2dfa-776a-4abe-bd8f-91b64d583d26 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:59.062755 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b557bc-9395-40dc-8696-202a0ffebcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:59.063189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b557bc-9395-40dc-8696-202a0ffebcb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:59.063479 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b557bc-9395-40dc-8696-202a0ffebcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:59.063760 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b557bc-9395-40dc-8696-202a0ffebcb5 admin admin] PATH_INFO: `/v3/projects/04aed805c6764e6191c4294e1fc90d7c/users/dff9858045a04d1e9a959e806c126e4c/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:59.086111 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b557bc-9395-40dc-8696-202a0ffebcb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04aed805c6764e6191c4294e1fc90d7c, user_id=dff9858045a04d1e9a959e806c126e4c, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:59.088043 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b557bc-9395-40dc-8696-202a0ffebcb5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:59.119884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1122/2226] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:59 2022] PUT /identity/v3/projects/04aed805c6764e6191c4294e1fc90d7c/users/dff9858045a04d1e9a959e806c126e4c/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:59.129703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107d4ca-7350-4cd6-a3bb-04ab74ad18cc None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:59.130568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107d4ca-7350-4cd6-a3bb-04ab74ad18cc None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:59.133730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107d4ca-7350-4cd6-a3bb-04ab74ad18cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:59.171918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e107d4ca-7350-4cd6-a3bb-04ab74ad18cc None None] MFA Rules not processed for user `dff9858045a04d1e9a959e806c126e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:53:59.173478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e107d4ca-7350-4cd6-a3bb-04ab74ad18cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:53:59.269804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1105/2227] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:53:59.270468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b557bc-9395-40dc-8696-202a0ffebcb5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:59.302110 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:59.364124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b557bc-9395-40dc-8696-202a0ffebcb5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:59.370547 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e84cc-7818-449b-a91e-26335d5627c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:59.371966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e84cc-7818-449b-a91e-26335d5627c9 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:59.371966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e84cc-7818-449b-a91e-26335d5627c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:59.371966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e84cc-7818-449b-a91e-26335d5627c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:59.372658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e84cc-7818-449b-a91e-26335d5627c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:59.374692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e84cc-7818-449b-a91e-26335d5627c9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:59.408468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1123/2228] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:53:59 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:59.410496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107d4ca-7350-4cd6-a3bb-04ab74ad18cc None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:59.419368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:53:59.447770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107d4ca-7350-4cd6-a3bb-04ab74ad18cc None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:59.529975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0b80c-965b-4d1a-8431-5f809e3aa429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:59.530669 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0b80c-965b-4d1a-8431-5f809e3aa429 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:59.530959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0b80c-965b-4d1a-8431-5f809e3aa429 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:59.531270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0b80c-965b-4d1a-8431-5f809e3aa429 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:59.531986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0b80c-965b-4d1a-8431-5f809e3aa429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:59.534230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0b80c-965b-4d1a-8431-5f809e3aa429 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:59.564525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1106/2229] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:53:59 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:53:59.568943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e84cc-7818-449b-a91e-26335d5627c9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:59.580463 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e84cc-7818-449b-a91e-26335d5627c9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:59.586462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755b2e6-99a3-4366-a53b-be3eef03de69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:59.587389 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755b2e6-99a3-4366-a53b-be3eef03de69 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:59.587661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755b2e6-99a3-4366-a53b-be3eef03de69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:59.588078 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755b2e6-99a3-4366-a53b-be3eef03de69 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:59.588859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755b2e6-99a3-4366-a53b-be3eef03de69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:59.590550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755b2e6-99a3-4366-a53b-be3eef03de69 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:59.605026 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1124/2230] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:53:59.611182 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0b80c-965b-4d1a-8431-5f809e3aa429 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:59.628802 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0b80c-965b-4d1a-8431-5f809e3aa429 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:59.636854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd51337-49d0-4621-85c0-b76be4fc1149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:59.637886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd51337-49d0-4621-85c0-b76be4fc1149 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:59.638362 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd51337-49d0-4621-85c0-b76be4fc1149 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:59.639591 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd51337-49d0-4621-85c0-b76be4fc1149 admin admin] PATH_INFO: `/v3/projects/c87831f1c8734b60803c36639e67f558/users/accdcb8e5273448389ded7f4e18e0a07/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:59.674194 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd51337-49d0-4621-85c0-b76be4fc1149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c87831f1c8734b60803c36639e67f558, user_id=accdcb8e5273448389ded7f4e18e0a07, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:59.677618 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd51337-49d0-4621-85c0-b76be4fc1149 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:59.710402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1107/2231] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:59 2022] PUT /identity/v3/projects/c87831f1c8734b60803c36639e67f558/users/accdcb8e5273448389ded7f4e18e0a07/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:59.715506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755b2e6-99a3-4366-a53b-be3eef03de69 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:59.730588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755b2e6-99a3-4366-a53b-be3eef03de69 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:59.853700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c72c8-da3b-4de0-9b16-b748a9c56f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:59.854416 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c72c8-da3b-4de0-9b16-b748a9c56f3d admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:59.854678 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c72c8-da3b-4de0-9b16-b748a9c56f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:59.854957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c72c8-da3b-4de0-9b16-b748a9c56f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:59.856056 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c72c8-da3b-4de0-9b16-b748a9c56f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:59.857447 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c72c8-da3b-4de0-9b16-b748a9c56f3d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:59.873121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1125/2232] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:53:59.877621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd51337-49d0-4621-85c0-b76be4fc1149 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:53:59.893676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd51337-49d0-4621-85c0-b76be4fc1149 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:53:59.899090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c0111-a976-41b9-a95a-cbab22e18e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:53:59.899913 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c0111-a976-41b9-a95a-cbab22e18e13 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:59.900310 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c0111-a976-41b9-a95a-cbab22e18e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:59.900657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c0111-a976-41b9-a95a-cbab22e18e13 admin admin] PATH_INFO: `/v3/projects/c87831f1c8734b60803c36639e67f558/users/accdcb8e5273448389ded7f4e18e0a07/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:53:59.903323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c0111-a976-41b9-a95a-cbab22e18e13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c87831f1c8734b60803c36639e67f558, user_id=accdcb8e5273448389ded7f4e18e0a07, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:53:59.908212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c0111-a976-41b9-a95a-cbab22e18e13 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:53:59.925648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1108/2233] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:53:59 2022] PUT /identity/v3/projects/c87831f1c8734b60803c36639e67f558/users/accdcb8e5273448389ded7f4e18e0a07/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:53:59.946156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b14764-8ee2-4c2f-a352-27801d074682 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:53:59.946711 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b14764-8ee2-4c2f-a352-27801d074682 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:53:59.948131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b14764-8ee2-4c2f-a352-27801d074682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:00.081651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-11b14764-8ee2-4c2f-a352-27801d074682 None None] MFA Rules not processed for user `accdcb8e5273448389ded7f4e18e0a07`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:00.092095 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-11b14764-8ee2-4c2f-a352-27801d074682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:00.280972 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1126/2234] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:53:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:00.291223 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c0111-a976-41b9-a95a-cbab22e18e13 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:00.293744 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c27c0111-a976-41b9-a95a-cbab22e18e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:00.373924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c0111-a976-41b9-a95a-cbab22e18e13 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:00.378482 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6285375-bf7f-4568-838a-2a4ef1ae8c68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:00.379734 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6285375-bf7f-4568-838a-2a4ef1ae8c68 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:00.380059 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6285375-bf7f-4568-838a-2a4ef1ae8c68 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:00.380336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6285375-bf7f-4568-838a-2a4ef1ae8c68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:00.476058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6285375-bf7f-4568-838a-2a4ef1ae8c68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:00.478128 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6285375-bf7f-4568-838a-2a4ef1ae8c68 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:00.488412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1109/2235] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:54:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:54:02.293043 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b14764-8ee2-4c2f-a352-27801d074682 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:02.296517 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-11b14764-8ee2-4c2f-a352-27801d074682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:02.313415 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b14764-8ee2-4c2f-a352-27801d074682 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:02.318977 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c744a-dd78-4729-8efa-aac0edf0706e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:02.324823 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c744a-dd78-4729-8efa-aac0edf0706e service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:02.326064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c744a-dd78-4729-8efa-aac0edf0706e service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:02.326640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c744a-dd78-4729-8efa-aac0edf0706e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:02.346509 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:02.426162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c744a-dd78-4729-8efa-aac0edf0706e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:02.428687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21c744a-dd78-4729-8efa-aac0edf0706e service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:02.439736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1127/2236] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:54:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:54:03.464871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6285375-bf7f-4568-838a-2a4ef1ae8c68 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:03.486102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6285375-bf7f-4568-838a-2a4ef1ae8c68 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:03.495094 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae295acf-b032-43e5-912b-14b9f1ea16e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:03.495850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae295acf-b032-43e5-912b-14b9f1ea16e7 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:03.496616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae295acf-b032-43e5-912b-14b9f1ea16e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:03.497112 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae295acf-b032-43e5-912b-14b9f1ea16e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:03.508665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae295acf-b032-43e5-912b-14b9f1ea16e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:03.511716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:03.513841 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae295acf-b032-43e5-912b-14b9f1ea16e7 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:03.540081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1110/2237] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:54:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:54:04.010697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b235008-fc89-4cfc-b61f-ce1c808538a7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:04.012530 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b235008-fc89-4cfc-b61f-ce1c808538a7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:04.017127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b235008-fc89-4cfc-b61f-ce1c808538a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:04.086375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3b235008-fc89-4cfc-b61f-ce1c808538a7 None None] MFA Rules not processed for user `73ef9d62f31d4135b6f079a26d791ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:04.087694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3b235008-fc89-4cfc-b61f-ce1c808538a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:04.187935 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1128/2238] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:04.211701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae295acf-b032-43e5-912b-14b9f1ea16e7 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:04.215027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ae295acf-b032-43e5-912b-14b9f1ea16e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:04.312738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae295acf-b032-43e5-912b-14b9f1ea16e7 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:04.327949 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4849b-a9ce-4abb-8ce3-96571ea3806a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:04.328676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4849b-a9ce-4abb-8ce3-96571ea3806a service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:04.328962 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4849b-a9ce-4abb-8ce3-96571ea3806a service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:04.329441 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4849b-a9ce-4abb-8ce3-96571ea3806a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:04.345496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4849b-a9ce-4abb-8ce3-96571ea3806a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:04.347867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4849b-a9ce-4abb-8ce3-96571ea3806a service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:04.362105 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1111/2239] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:54:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:54:06.305872 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a2a6a-60f2-46b6-8c5b-47c80db06f23 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:06.307249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a2a6a-60f2-46b6-8c5b-47c80db06f23 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:06.307756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a2a6a-60f2-46b6-8c5b-47c80db06f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:06.351620 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4d5a2a6a-60f2-46b6-8c5b-47c80db06f23 None None] MFA Rules not processed for user `73ef9d62f31d4135b6f079a26d791ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:06.353249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4d5a2a6a-60f2-46b6-8c5b-47c80db06f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:06.361205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1129/2240] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:06.689088 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4849b-a9ce-4abb-8ce3-96571ea3806a service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:06.692346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-05e4849b-a9ce-4abb-8ce3-96571ea3806a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:06.786077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4849b-a9ce-4abb-8ce3-96571ea3806a service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:06.795128 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7aa46-b895-4b35-98c3-69b417d61bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:06.795978 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7aa46-b895-4b35-98c3-69b417d61bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:06.796424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7aa46-b895-4b35-98c3-69b417d61bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:06.796707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb7aa46-b895-4b35-98c3-69b417d61bd2 admin admin] PATH_INFO: `/v3/users/0cceaddf5fb045f5a3e2de8e3c0be216` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:06.799216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7aa46-b895-4b35-98c3-69b417d61bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cceaddf5fb045f5a3e2de8e3c0be216)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:06.804092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb7aa46-b895-4b35-98c3-69b417d61bd2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:06.883725 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ccb7aa46-b895-4b35-98c3-69b417d61bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cceaddf5fb045f5a3e2de8e3c0be216'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:06.898239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ccb7aa46-b895-4b35-98c3-69b417d61bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cceaddf5fb045f5a3e2de8e3c0be216'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:06.935900 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ccb7aa46-b895-4b35-98c3-69b417d61bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cceaddf5fb045f5a3e2de8e3c0be216'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:06.960216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ccb7aa46-b895-4b35-98c3-69b417d61bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cceaddf5fb045f5a3e2de8e3c0be216'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:06.960852 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ccb7aa46-b895-4b35-98c3-69b417d61bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cceaddf5fb045f5a3e2de8e3c0be216'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:06.964457 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:06.977442 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1112/2241] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:54:06 2022] DELETE /identity/v3/users/0cceaddf5fb045f5a3e2de8e3c0be216 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:54:06.979718 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7995b-842c-413c-bb77-8ecd6c0627a9 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:06.981319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7995b-842c-413c-bb77-8ecd6c0627a9 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:06.981664 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7995b-842c-413c-bb77-8ecd6c0627a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:07.005710 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d5e7995b-842c-413c-bb77-8ecd6c0627a9 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:07.007091 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d5e7995b-842c-413c-bb77-8ecd6c0627a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:07.113107 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1130/2242] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:07.136738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7aa46-b895-4b35-98c3-69b417d61bd2 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:07.138576 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ccb7aa46-b895-4b35-98c3-69b417d61bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:07.241973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb7aa46-b895-4b35-98c3-69b417d61bd2 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:07.247996 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2b19a-868b-4d97-8e70-9ad0335ba015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:07.254347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2b19a-868b-4d97-8e70-9ad0335ba015 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:07.254845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2b19a-868b-4d97-8e70-9ad0335ba015 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:07.255148 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2b19a-868b-4d97-8e70-9ad0335ba015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:07.285929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2b19a-868b-4d97-8e70-9ad0335ba015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:07.287444 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2b19a-868b-4d97-8e70-9ad0335ba015 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:07.310934 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1113/2243] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:54:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:54:07.728254 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7995b-842c-413c-bb77-8ecd6c0627a9 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:07.730086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d5e7995b-842c-413c-bb77-8ecd6c0627a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:07.758668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7995b-842c-413c-bb77-8ecd6c0627a9 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:07.789261 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c1ea6-8db5-4536-825f-8f35042eb9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:07.790774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c1ea6-8db5-4536-825f-8f35042eb9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:07.791064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c1ea6-8db5-4536-825f-8f35042eb9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:07.791522 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c1ea6-8db5-4536-825f-8f35042eb9d0 admin admin] PATH_INFO: `/v3/projects/ab18ed20453340a59ab3732de6483034` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:07.795119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c1ea6-8db5-4536-825f-8f35042eb9d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab18ed20453340a59ab3732de6483034)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:07.800022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c1ea6-8db5-4536-825f-8f35042eb9d0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:07.856173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:07.856748 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2b19a-868b-4d97-8e70-9ad0335ba015 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:07.860514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-fbd2b19a-868b-4d97-8e70-9ad0335ba015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:07.874995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e52c1ea6-8db5-4536-825f-8f35042eb9d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab18ed20453340a59ab3732de6483034', 'request_id': 'req-e52c1ea6-8db5-4536-825f-8f35042eb9d0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:07.878531 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2b19a-868b-4d97-8e70-9ad0335ba015 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:07.887449 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e52c1ea6-8db5-4536-825f-8f35042eb9d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab18ed20453340a59ab3732de6483034', 'request_id': 'req-e52c1ea6-8db5-4536-825f-8f35042eb9d0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:07.896301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e52c1ea6-8db5-4536-825f-8f35042eb9d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab18ed20453340a59ab3732de6483034', 'request_id': 'req-e52c1ea6-8db5-4536-825f-8f35042eb9d0'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:07.914430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e52c1ea6-8db5-4536-825f-8f35042eb9d0 admin admin] The token cache is being invalidate because project ab18ed20453340a59ab3732de6483034 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:54:07.914998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e52c1ea6-8db5-4536-825f-8f35042eb9d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:07.915802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e52c1ea6-8db5-4536-825f-8f35042eb9d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:07.917362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1131/2244] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:54:07 2022] DELETE /identity/v3/projects/ab18ed20453340a59ab3732de6483034 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:54:07.965569 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4156b-8d2b-47f5-a224-e1f42aec72ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:07.966127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4156b-8d2b-47f5-a224-e1f42aec72ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:07.966411 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4156b-8d2b-47f5-a224-e1f42aec72ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:07.966871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4156b-8d2b-47f5-a224-e1f42aec72ab admin admin] PATH_INFO: `/v3/users/017efbc4c7684f48a9893b12f6fd19d4` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:07.968364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4156b-8d2b-47f5-a224-e1f42aec72ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=017efbc4c7684f48a9893b12f6fd19d4)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:07.970406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4156b-8d2b-47f5-a224-e1f42aec72ab admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:08.063432 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-80a4156b-8d2b-47f5-a224-e1f42aec72ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '017efbc4c7684f48a9893b12f6fd19d4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:08.091565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-80a4156b-8d2b-47f5-a224-e1f42aec72ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '017efbc4c7684f48a9893b12f6fd19d4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:08.119079 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-80a4156b-8d2b-47f5-a224-e1f42aec72ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '017efbc4c7684f48a9893b12f6fd19d4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:08.134480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-80a4156b-8d2b-47f5-a224-e1f42aec72ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '017efbc4c7684f48a9893b12f6fd19d4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:08.136257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-80a4156b-8d2b-47f5-a224-e1f42aec72ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '017efbc4c7684f48a9893b12f6fd19d4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:08.143599 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1114/2245] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:54:07 2022] DELETE /identity/v3/users/017efbc4c7684f48a9893b12f6fd19d4 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:54:08.149784 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79816149-0212-43af-8ee9-c1128212b3a5 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:08.152917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79816149-0212-43af-8ee9-c1128212b3a5 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:08.153266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79816149-0212-43af-8ee9-c1128212b3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:08.174345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-79816149-0212-43af-8ee9-c1128212b3a5 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:08.175567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-79816149-0212-43af-8ee9-c1128212b3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:08.268392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1132/2246] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:08.275702 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4156b-8d2b-47f5-a224-e1f42aec72ab admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:08.277361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-80a4156b-8d2b-47f5-a224-e1f42aec72ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:08.316841 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:08.385424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4156b-8d2b-47f5-a224-e1f42aec72ab admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:08.390913 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d4c9e-5b61-4290-82b8-96531a8072dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:08.392688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d4c9e-5b61-4290-82b8-96531a8072dd service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:08.392969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d4c9e-5b61-4290-82b8-96531a8072dd service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:08.393255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d4c9e-5b61-4290-82b8-96531a8072dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:08.405060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d4c9e-5b61-4290-82b8-96531a8072dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:08.408923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d4c9e-5b61-4290-82b8-96531a8072dd service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:08.420104 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1115/2247] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:54:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:54:08.924527 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79816149-0212-43af-8ee9-c1128212b3a5 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:08.926785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-79816149-0212-43af-8ee9-c1128212b3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:08.948909 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79816149-0212-43af-8ee9-c1128212b3a5 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:08.976338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6b698-929d-4ecc-b808-72fe29de1b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:08.977457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6b698-929d-4ecc-b808-72fe29de1b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:08.978413 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6b698-929d-4ecc-b808-72fe29de1b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:08.978837 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6b698-929d-4ecc-b808-72fe29de1b3f admin admin] PATH_INFO: `/v3/projects/1656aecf459c46919e0aee846f064586` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:08.982606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6b698-929d-4ecc-b808-72fe29de1b3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1656aecf459c46919e0aee846f064586)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:08.986355 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6b698-929d-4ecc-b808-72fe29de1b3f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:09.080840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bee6b698-929d-4ecc-b808-72fe29de1b3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1656aecf459c46919e0aee846f064586', 'request_id': 'req-bee6b698-929d-4ecc-b808-72fe29de1b3f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:09.096613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bee6b698-929d-4ecc-b808-72fe29de1b3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1656aecf459c46919e0aee846f064586', 'request_id': 'req-bee6b698-929d-4ecc-b808-72fe29de1b3f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:09.109676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bee6b698-929d-4ecc-b808-72fe29de1b3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1656aecf459c46919e0aee846f064586', 'request_id': 'req-bee6b698-929d-4ecc-b808-72fe29de1b3f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:09.130050 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bee6b698-929d-4ecc-b808-72fe29de1b3f admin admin] The token cache is being invalidate because project 1656aecf459c46919e0aee846f064586 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:54:09.134612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bee6b698-929d-4ecc-b808-72fe29de1b3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:09.136396 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bee6b698-929d-4ecc-b808-72fe29de1b3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:09.141248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1133/2248] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:54:08 2022] DELETE /identity/v3/projects/1656aecf459c46919e0aee846f064586 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:54:09.142761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d4c9e-5b61-4290-82b8-96531a8072dd service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:09.156044 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d4c9e-5b61-4290-82b8-96531a8072dd service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:09.238085 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98da636-bd38-4545-afd4-98afdd991d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:09.240389 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98da636-bd38-4545-afd4-98afdd991d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:09.240805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98da636-bd38-4545-afd4-98afdd991d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:09.241347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98da636-bd38-4545-afd4-98afdd991d9c admin admin] PATH_INFO: `/v3/users/dff9858045a04d1e9a959e806c126e4c` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:09.243111 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98da636-bd38-4545-afd4-98afdd991d9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dff9858045a04d1e9a959e806c126e4c)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:09.245049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98da636-bd38-4545-afd4-98afdd991d9c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:09.308195 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d98da636-bd38-4545-afd4-98afdd991d9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dff9858045a04d1e9a959e806c126e4c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:09.317475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d98da636-bd38-4545-afd4-98afdd991d9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dff9858045a04d1e9a959e806c126e4c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:09.334307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d98da636-bd38-4545-afd4-98afdd991d9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dff9858045a04d1e9a959e806c126e4c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:09.343772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d98da636-bd38-4545-afd4-98afdd991d9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dff9858045a04d1e9a959e806c126e4c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:09.344617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d98da636-bd38-4545-afd4-98afdd991d9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dff9858045a04d1e9a959e806c126e4c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:09.355448 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1116/2249] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:54:09 2022] DELETE /identity/v3/users/dff9858045a04d1e9a959e806c126e4c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:54:09.663217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6b698-929d-4ecc-b808-72fe29de1b3f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:09.665055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bee6b698-929d-4ecc-b808-72fe29de1b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:09.687697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6b698-929d-4ecc-b808-72fe29de1b3f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:09.755661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:09.774675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb221f4-7c43-47c9-8600-c9727b35ea7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:09.775393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb221f4-7c43-47c9-8600-c9727b35ea7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:09.775802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb221f4-7c43-47c9-8600-c9727b35ea7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:09.776212 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb221f4-7c43-47c9-8600-c9727b35ea7f admin admin] PATH_INFO: `/v3/projects/04aed805c6764e6191c4294e1fc90d7c` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:09.778012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb221f4-7c43-47c9-8600-c9727b35ea7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=04aed805c6764e6191c4294e1fc90d7c)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:09.780112 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb221f4-7c43-47c9-8600-c9727b35ea7f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:09.859993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ddb221f4-7c43-47c9-8600-c9727b35ea7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04aed805c6764e6191c4294e1fc90d7c', 'request_id': 'req-ddb221f4-7c43-47c9-8600-c9727b35ea7f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:09.874539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ddb221f4-7c43-47c9-8600-c9727b35ea7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04aed805c6764e6191c4294e1fc90d7c', 'request_id': 'req-ddb221f4-7c43-47c9-8600-c9727b35ea7f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:09.884017 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ddb221f4-7c43-47c9-8600-c9727b35ea7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04aed805c6764e6191c4294e1fc90d7c', 'request_id': 'req-ddb221f4-7c43-47c9-8600-c9727b35ea7f'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:09.916896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ddb221f4-7c43-47c9-8600-c9727b35ea7f admin admin] The token cache is being invalidate because project 04aed805c6764e6191c4294e1fc90d7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:54:09.917640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ddb221f4-7c43-47c9-8600-c9727b35ea7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:09.918565 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ddb221f4-7c43-47c9-8600-c9727b35ea7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:09.923045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1134/2250] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:54:09 2022] DELETE /identity/v3/projects/04aed805c6764e6191c4294e1fc90d7c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:54:09.929900 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98da636-bd38-4545-afd4-98afdd991d9c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:09.931655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d98da636-bd38-4545-afd4-98afdd991d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:09.948100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98da636-bd38-4545-afd4-98afdd991d9c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:10.012494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:10.038832 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aacbc3-0374-4d2c-b536-f3cada563b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:10.039392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aacbc3-0374-4d2c-b536-f3cada563b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:10.039671 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aacbc3-0374-4d2c-b536-f3cada563b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:10.040013 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aacbc3-0374-4d2c-b536-f3cada563b28 admin admin] PATH_INFO: `/v3/users/accdcb8e5273448389ded7f4e18e0a07` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:10.041246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aacbc3-0374-4d2c-b536-f3cada563b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=accdcb8e5273448389ded7f4e18e0a07)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:10.043425 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aacbc3-0374-4d2c-b536-f3cada563b28 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:10.110104 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-21aacbc3-0374-4d2c-b536-f3cada563b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'accdcb8e5273448389ded7f4e18e0a07'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:10.117575 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-21aacbc3-0374-4d2c-b536-f3cada563b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'accdcb8e5273448389ded7f4e18e0a07'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:10.135319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-21aacbc3-0374-4d2c-b536-f3cada563b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'accdcb8e5273448389ded7f4e18e0a07'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:10.145271 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-21aacbc3-0374-4d2c-b536-f3cada563b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'accdcb8e5273448389ded7f4e18e0a07'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:10.146568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-21aacbc3-0374-4d2c-b536-f3cada563b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'accdcb8e5273448389ded7f4e18e0a07'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:10.154575 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1117/2251] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:54:09 2022] DELETE /identity/v3/users/accdcb8e5273448389ded7f4e18e0a07 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:54:10.446797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb221f4-7c43-47c9-8600-c9727b35ea7f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:10.448291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ddb221f4-7c43-47c9-8600-c9727b35ea7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:10.467641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb221f4-7c43-47c9-8600-c9727b35ea7f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:10.534598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5763e58-37ee-40a7-9b8d-5450da213d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:10.535265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5763e58-37ee-40a7-9b8d-5450da213d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:10.535544 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5763e58-37ee-40a7-9b8d-5450da213d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:10.535840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5763e58-37ee-40a7-9b8d-5450da213d2e admin admin] PATH_INFO: `/v3/projects/c87831f1c8734b60803c36639e67f558` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:10.537170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5763e58-37ee-40a7-9b8d-5450da213d2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c87831f1c8734b60803c36639e67f558)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:10.539119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5763e58-37ee-40a7-9b8d-5450da213d2e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:10.597364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e5763e58-37ee-40a7-9b8d-5450da213d2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c87831f1c8734b60803c36639e67f558', 'request_id': 'req-e5763e58-37ee-40a7-9b8d-5450da213d2e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:10.606064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e5763e58-37ee-40a7-9b8d-5450da213d2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c87831f1c8734b60803c36639e67f558', 'request_id': 'req-e5763e58-37ee-40a7-9b8d-5450da213d2e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:10.612957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e5763e58-37ee-40a7-9b8d-5450da213d2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c87831f1c8734b60803c36639e67f558', 'request_id': 'req-e5763e58-37ee-40a7-9b8d-5450da213d2e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:10.632180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e5763e58-37ee-40a7-9b8d-5450da213d2e admin admin] The token cache is being invalidate because project c87831f1c8734b60803c36639e67f558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:54:10.632759 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e5763e58-37ee-40a7-9b8d-5450da213d2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:10.633528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e5763e58-37ee-40a7-9b8d-5450da213d2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:10.635346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1135/2252] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:54:10 2022] DELETE /identity/v3/projects/c87831f1c8734b60803c36639e67f558 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:54:10.641588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4047574-9842-429a-9830-73627a7e5d7b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:10.642002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4047574-9842-429a-9830-73627a7e5d7b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:10.642376 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4047574-9842-429a-9830-73627a7e5d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:10.664208 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e4047574-9842-429a-9830-73627a7e5d7b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:10.665434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e4047574-9842-429a-9830-73627a7e5d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:10.746456 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1118/2253] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:54:10.754526 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b9b6c-0507-4b93-bb71-4a8e780cb6dd None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:10.754922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b9b6c-0507-4b93-bb71-4a8e780cb6dd None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:10.755999 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b9b6c-0507-4b93-bb71-4a8e780cb6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:10.766632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:10.787690 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-093b9b6c-0507-4b93-bb71-4a8e780cb6dd None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:10.789259 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-093b9b6c-0507-4b93-bb71-4a8e780cb6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:10.807003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1136/2254] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:10.810977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4047574-9842-429a-9830-73627a7e5d7b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:10.823445 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4047574-9842-429a-9830-73627a7e5d7b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:10.832857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2947e59-8b3d-4427-99ae-964be689847a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:10.833539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2947e59-8b3d-4427-99ae-964be689847a admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:10.833822 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2947e59-8b3d-4427-99ae-964be689847a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:10.834093 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2947e59-8b3d-4427-99ae-964be689847a admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:10.835068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2947e59-8b3d-4427-99ae-964be689847a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:54:10.835430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2947e59-8b3d-4427-99ae-964be689847a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:10.837106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2947e59-8b3d-4427-99ae-964be689847a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:10.855769 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b9b6c-0507-4b93-bb71-4a8e780cb6dd None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:10.866482 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b9b6c-0507-4b93-bb71-4a8e780cb6dd None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:10.867400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1119/2255] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:54:10.872127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a370d9-9adf-47ff-aefd-1d3565facf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:10.872748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a370d9-9adf-47ff-aefd-1d3565facf74 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:10.873026 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a370d9-9adf-47ff-aefd-1d3565facf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:10.873328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a370d9-9adf-47ff-aefd-1d3565facf74 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:10.874156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a370d9-9adf-47ff-aefd-1d3565facf74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:10.875935 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a370d9-9adf-47ff-aefd-1d3565facf74 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:10.911229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1137/2256] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:54:10 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:54:10.915184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2947e59-8b3d-4427-99ae-964be689847a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:10.925578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2947e59-8b3d-4427-99ae-964be689847a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:10.995011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:11.018497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f90ae-6e2a-4e64-89a8-5da867c14b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:11.019412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f90ae-6e2a-4e64-89a8-5da867c14b97 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:11.019860 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f90ae-6e2a-4e64-89a8-5da867c14b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:11.020311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f90ae-6e2a-4e64-89a8-5da867c14b97 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:11.021579 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f90ae-6e2a-4e64-89a8-5da867c14b97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:11.023708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f90ae-6e2a-4e64-89a8-5da867c14b97 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:11.053247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1120/2257] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:54:10 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:54:11.057311 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a370d9-9adf-47ff-aefd-1d3565facf74 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:11.080429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a370d9-9adf-47ff-aefd-1d3565facf74 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:11.098302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f14d7e-4ade-4df6-ba4a-dde4c013c1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:11.100207 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f14d7e-4ade-4df6-ba4a-dde4c013c1ac admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:11.100421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f14d7e-4ade-4df6-ba4a-dde4c013c1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:11.100848 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f14d7e-4ade-4df6-ba4a-dde4c013c1ac admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:11.101506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f14d7e-4ade-4df6-ba4a-dde4c013c1ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:11.104013 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f14d7e-4ade-4df6-ba4a-dde4c013c1ac admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:11.126021 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f90ae-6e2a-4e64-89a8-5da867c14b97 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:11.127025 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1138/2258] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:54:11.139929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f90ae-6e2a-4e64-89a8-5da867c14b97 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:11.145242 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8cc43-62cc-4352-b94a-724a25a9e251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:11.146031 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8cc43-62cc-4352-b94a-724a25a9e251 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:11.146606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8cc43-62cc-4352-b94a-724a25a9e251 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:11.146887 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8cc43-62cc-4352-b94a-724a25a9e251 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:11.156723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8cc43-62cc-4352-b94a-724a25a9e251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:11.158562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8cc43-62cc-4352-b94a-724a25a9e251 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:11.174744 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1121/2259] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:54:11 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:54:11.178969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f14d7e-4ade-4df6-ba4a-dde4c013c1ac admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:11.189991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f14d7e-4ade-4df6-ba4a-dde4c013c1ac admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:11.195009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2171817-5a37-4d59-8c8f-9547aef1e54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:11.195678 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2171817-5a37-4d59-8c8f-9547aef1e54f admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:11.195989 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2171817-5a37-4d59-8c8f-9547aef1e54f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:11.196260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2171817-5a37-4d59-8c8f-9547aef1e54f admin admin] PATH_INFO: `/v3/projects/431f4daf59224848a25db385c7541c95/users/f6485aba952244fcac8d0898e229bc83/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:11.211739 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2171817-5a37-4d59-8c8f-9547aef1e54f admin admin] RBAC: Authorizing `identity:create_grant(project_id=431f4daf59224848a25db385c7541c95, user_id=f6485aba952244fcac8d0898e229bc83, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:11.214692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2171817-5a37-4d59-8c8f-9547aef1e54f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:11.229356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1139/2260] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:54:11 2022] PUT /identity/v3/projects/431f4daf59224848a25db385c7541c95/users/f6485aba952244fcac8d0898e229bc83/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:54:11.241641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d8a84-2599-44e9-9c97-32a782474cf5 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:11.241930 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d8a84-2599-44e9-9c97-32a782474cf5 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:11.242211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d8a84-2599-44e9-9c97-32a782474cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:11.273040 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-dc3d8a84-2599-44e9-9c97-32a782474cf5 None None] MFA Rules not processed for user `f6485aba952244fcac8d0898e229bc83`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:11.274932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dc3d8a84-2599-44e9-9c97-32a782474cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:11.353148 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1122/2261] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:54:11.364395 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2171817-5a37-4d59-8c8f-9547aef1e54f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:11.366181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f2171817-5a37-4d59-8c8f-9547aef1e54f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:11.430946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2171817-5a37-4d59-8c8f-9547aef1e54f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:11.434929 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce003ead-0be8-4adb-a42d-cdff8c6055e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:11.435545 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce003ead-0be8-4adb-a42d-cdff8c6055e6 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:11.435851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce003ead-0be8-4adb-a42d-cdff8c6055e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:11.436124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce003ead-0be8-4adb-a42d-cdff8c6055e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:11.445982 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce003ead-0be8-4adb-a42d-cdff8c6055e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:11.447424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce003ead-0be8-4adb-a42d-cdff8c6055e6 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:11.457391 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1140/2262] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:54:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:54:12.823940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2add0d-61ab-4aad-bfa1-28f7a6c7c98d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:12.824491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2add0d-61ab-4aad-bfa1-28f7a6c7c98d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:12.824491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2add0d-61ab-4aad-bfa1-28f7a6c7c98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:12.843179 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8c2add0d-61ab-4aad-bfa1-28f7a6c7c98d None None] MFA Rules not processed for user `f6485aba952244fcac8d0898e229bc83`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:12.844441 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8c2add0d-61ab-4aad-bfa1-28f7a6c7c98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:12.852784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1123/2263] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:21.453966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:22.837949 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:29.627240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e8239-b8a0-4c01-b79d-8ca5013b7679 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:29.627733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e8239-b8a0-4c01-b79d-8ca5013b7679 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:29.627891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e8239-b8a0-4c01-b79d-8ca5013b7679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:29.648883 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1b7e8239-b8a0-4c01-b79d-8ca5013b7679 None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:29.650075 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1b7e8239-b8a0-4c01-b79d-8ca5013b7679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:29.709336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1141/2264] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Sep 19 14:54:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:54:29.753039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2add0d-61ab-4aad-bfa1-28f7a6c7c98d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:29.754663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8c2add0d-61ab-4aad-bfa1-28f7a6c7c98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:29.833760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2add0d-61ab-4aad-bfa1-28f7a6c7c98d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:29.838734 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a830f-a45a-4d75-a2e1-6b2ba70dce35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:29.839333 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a830f-a45a-4d75-a2e1-6b2ba70dce35 service swift] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:29.839641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a830f-a45a-4d75-a2e1-6b2ba70dce35 service swift] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:29.839943 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a830f-a45a-4d75-a2e1-6b2ba70dce35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:29.850057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a830f-a45a-4d75-a2e1-6b2ba70dce35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:29.851651 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a830f-a45a-4d75-a2e1-6b2ba70dce35 service swift] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:29.860517 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1124/2265] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 14:54:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 14:54:30.403218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445316c-90e9-46d2-ab4a-4a6b9bc16f77 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:30.403418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445316c-90e9-46d2-ab4a-4a6b9bc16f77 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:30.403724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445316c-90e9-46d2-ab4a-4a6b9bc16f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:30.423999 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b445316c-90e9-46d2-ab4a-4a6b9bc16f77 None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:30.425218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b445316c-90e9-46d2-ab4a-4a6b9bc16f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:30.433188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1142/2266] 10.222.0.60 () {58 vars in 967 bytes} [Mon Sep 19 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:30.447149 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a830f-a45a-4d75-a2e1-6b2ba70dce35 service swift] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:30.457612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a830f-a45a-4d75-a2e1-6b2ba70dce35 service swift] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:30.463153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfe466-6277-42a1-82b5-806e57bf10b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:30.463747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfe466-6277-42a1-82b5-806e57bf10b5 service swift] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:30.464062 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfe466-6277-42a1-82b5-806e57bf10b5 service swift] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:30.464337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdfe466-6277-42a1-82b5-806e57bf10b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:30.472773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfe466-6277-42a1-82b5-806e57bf10b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:30.474387 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdfe466-6277-42a1-82b5-806e57bf10b5 service swift] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:30.482989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1125/2267] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 14:54:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 14:54:30.742291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613ae6b-33f2-405b-8c9a-d5b5b7839c19 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:30.742806 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613ae6b-33f2-405b-8c9a-d5b5b7839c19 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:30.742935 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e613ae6b-33f2-405b-8c9a-d5b5b7839c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:30.761926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e613ae6b-33f2-405b-8c9a-d5b5b7839c19 None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:30.763105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e613ae6b-33f2-405b-8c9a-d5b5b7839c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:30.770086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1143/2268] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Sep 19 14:54:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:30.782492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfe466-6277-42a1-82b5-806e57bf10b5 service swift] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:30.793319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdfe466-6277-42a1-82b5-806e57bf10b5 service swift] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:30.798786 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25c2b1-7bad-4eed-b533-327ff346d49e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:30.799382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25c2b1-7bad-4eed-b533-327ff346d49e service swift] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:30.799671 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25c2b1-7bad-4eed-b533-327ff346d49e service swift] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:30.799981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb25c2b1-7bad-4eed-b533-327ff346d49e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:30.809003 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25c2b1-7bad-4eed-b533-327ff346d49e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:30.810615 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb25c2b1-7bad-4eed-b533-327ff346d49e service swift] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:30.821399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1126/2269] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 14:54:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 14:54:31.175127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bfe7b-b646-4480-8cb1-198d76250297 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:31.175425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bfe7b-b646-4480-8cb1-198d76250297 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:31.175695 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5bfe7b-b646-4480-8cb1-198d76250297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:31.195272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-7b5bfe7b-b646-4480-8cb1-198d76250297 None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:31.196456 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7b5bfe7b-b646-4480-8cb1-198d76250297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:31.203704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1144/2270] 10.222.0.60 () {58 vars in 967 bytes} [Mon Sep 19 14:54:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:31.216984 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25c2b1-7bad-4eed-b533-327ff346d49e service swift] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:31.227516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb25c2b1-7bad-4eed-b533-327ff346d49e service swift] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:31.232741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603303c-45e4-4a03-bd7b-643d8c32cde9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:31.233217 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603303c-45e4-4a03-bd7b-643d8c32cde9 service swift] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:31.233498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603303c-45e4-4a03-bd7b-643d8c32cde9 service swift] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:31.233772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603303c-45e4-4a03-bd7b-643d8c32cde9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:31.241732 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603303c-45e4-4a03-bd7b-643d8c32cde9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:31.243318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603303c-45e4-4a03-bd7b-643d8c32cde9 service swift] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:31.251261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1127/2271] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 14:54:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 14:54:33.377940 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bfe7b-b646-4480-8cb1-198d76250297 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:33.441806 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5bfe7b-b646-4480-8cb1-198d76250297 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:33.446863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f2098-8e37-4dd6-898a-0a7ebe5bcd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:33.447487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f2098-8e37-4dd6-898a-0a7ebe5bcd70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:33.447765 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f2098-8e37-4dd6-898a-0a7ebe5bcd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:33.448088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f2098-8e37-4dd6-898a-0a7ebe5bcd70 admin admin] PATH_INFO: `/v3/users/f6485aba952244fcac8d0898e229bc83` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:33.449202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f2098-8e37-4dd6-898a-0a7ebe5bcd70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6485aba952244fcac8d0898e229bc83)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:33.451085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f2098-8e37-4dd6-898a-0a7ebe5bcd70 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:33.500664 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8e0f2098-8e37-4dd6-898a-0a7ebe5bcd70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6485aba952244fcac8d0898e229bc83'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:33.508388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8e0f2098-8e37-4dd6-898a-0a7ebe5bcd70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6485aba952244fcac8d0898e229bc83'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:33.520293 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8e0f2098-8e37-4dd6-898a-0a7ebe5bcd70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6485aba952244fcac8d0898e229bc83'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:33.528636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8e0f2098-8e37-4dd6-898a-0a7ebe5bcd70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6485aba952244fcac8d0898e229bc83'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:33.529389 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-8e0f2098-8e37-4dd6-898a-0a7ebe5bcd70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6485aba952244fcac8d0898e229bc83'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:33.535291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1145/2272] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:54:33 2022] DELETE /identity/v3/users/f6485aba952244fcac8d0898e229bc83 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:54:33.540178 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c798bde-ebe8-429f-933e-23e860787518 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:33.540773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c798bde-ebe8-429f-933e-23e860787518 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:33.540773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c798bde-ebe8-429f-933e-23e860787518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:33.560412 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1c798bde-ebe8-429f-933e-23e860787518 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:33.561631 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1c798bde-ebe8-429f-933e-23e860787518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:33.627319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1128/2273] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:33.637422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f2098-8e37-4dd6-898a-0a7ebe5bcd70 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:33.638858 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8e0f2098-8e37-4dd6-898a-0a7ebe5bcd70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:33.645365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:33.697746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f2098-8e37-4dd6-898a-0a7ebe5bcd70 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:33.702312 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa78451-ab04-4c30-9a35-8fad5c4a6281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:33.703135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa78451-ab04-4c30-9a35-8fad5c4a6281 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:33.703507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa78451-ab04-4c30-9a35-8fad5c4a6281 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:33.703777 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa78451-ab04-4c30-9a35-8fad5c4a6281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:33.713940 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa78451-ab04-4c30-9a35-8fad5c4a6281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:33.715393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa78451-ab04-4c30-9a35-8fad5c4a6281 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:33.724881 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1146/2274] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:54:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:54:33.899384 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c798bde-ebe8-429f-933e-23e860787518 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:33.901374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1c798bde-ebe8-429f-933e-23e860787518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:33.918128 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c798bde-ebe8-429f-933e-23e860787518 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:33.923490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ff782-0e23-4a7b-a917-f6e6531d4c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:33.924277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ff782-0e23-4a7b-a917-f6e6531d4c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:33.924621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ff782-0e23-4a7b-a917-f6e6531d4c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:33.924950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ff782-0e23-4a7b-a917-f6e6531d4c40 admin admin] PATH_INFO: `/v3/projects/431f4daf59224848a25db385c7541c95` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:33.926311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ff782-0e23-4a7b-a917-f6e6531d4c40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=431f4daf59224848a25db385c7541c95)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:33.928362 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ff782-0e23-4a7b-a917-f6e6531d4c40 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:33.985836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:33.987752 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d15ff782-0e23-4a7b-a917-f6e6531d4c40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '431f4daf59224848a25db385c7541c95', 'request_id': 'req-d15ff782-0e23-4a7b-a917-f6e6531d4c40'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:33.997640 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d15ff782-0e23-4a7b-a917-f6e6531d4c40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '431f4daf59224848a25db385c7541c95', 'request_id': 'req-d15ff782-0e23-4a7b-a917-f6e6531d4c40'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:34.016765 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d15ff782-0e23-4a7b-a917-f6e6531d4c40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '431f4daf59224848a25db385c7541c95', 'request_id': 'req-d15ff782-0e23-4a7b-a917-f6e6531d4c40'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:34.030068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d15ff782-0e23-4a7b-a917-f6e6531d4c40 admin admin] The token cache is being invalidate because project 431f4daf59224848a25db385c7541c95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:54:34.030368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d15ff782-0e23-4a7b-a917-f6e6531d4c40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:34.031012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-d15ff782-0e23-4a7b-a917-f6e6531d4c40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:54:34.032554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1129/2275] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:54:33 2022] DELETE /identity/v3/projects/431f4daf59224848a25db385c7541c95 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:54:34.038142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4870090-a105-495e-8656-ce5e02b35028 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:34.038328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4870090-a105-495e-8656-ce5e02b35028 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:34.038587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4870090-a105-495e-8656-ce5e02b35028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:34.059643 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a4870090-a105-495e-8656-ce5e02b35028 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:34.060922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a4870090-a105-495e-8656-ce5e02b35028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:34.127657 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1147/2276] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:34.135981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f27eeb-6ec6-4cfb-bd80-fb55bac2e965 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:34.136274 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f27eeb-6ec6-4cfb-bd80-fb55bac2e965 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:34.136555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f27eeb-6ec6-4cfb-bd80-fb55bac2e965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:34.159996 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-46f27eeb-6ec6-4cfb-bd80-fb55bac2e965 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:34.161204 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-46f27eeb-6ec6-4cfb-bd80-fb55bac2e965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:34.169337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1130/2277] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:34.173715 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4870090-a105-495e-8656-ce5e02b35028 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:34.183948 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4870090-a105-495e-8656-ce5e02b35028 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:34.188689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c3756-b9c0-42f0-a526-a1acd6b2f202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:34.189288 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c3756-b9c0-42f0-a526-a1acd6b2f202 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:34.189570 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c3756-b9c0-42f0-a526-a1acd6b2f202 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:34.189844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c3756-b9c0-42f0-a526-a1acd6b2f202 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:34.190484 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c3756-b9c0-42f0-a526-a1acd6b2f202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:54:34.190786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c3756-b9c0-42f0-a526-a1acd6b2f202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:34.192440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c3756-b9c0-42f0-a526-a1acd6b2f202 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:34.204592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1148/2278] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:54:34.208868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f27eeb-6ec6-4cfb-bd80-fb55bac2e965 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:34.218725 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f27eeb-6ec6-4cfb-bd80-fb55bac2e965 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:34.223370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a0f6e-dcef-498b-86b6-4abfbe7647fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:34.224001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a0f6e-dcef-498b-86b6-4abfbe7647fd admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:34.224285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a0f6e-dcef-498b-86b6-4abfbe7647fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:34.224558 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a0f6e-dcef-498b-86b6-4abfbe7647fd admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:34.225192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a0f6e-dcef-498b-86b6-4abfbe7647fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:34.227089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a0f6e-dcef-498b-86b6-4abfbe7647fd admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:34.253967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1131/2279] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:54:34 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:54:34.258628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c3756-b9c0-42f0-a526-a1acd6b2f202 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:34.273410 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c3756-b9c0-42f0-a526-a1acd6b2f202 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:34.353791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc75ea-0f39-4354-b11a-c0202b10d9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:34.354419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc75ea-0f39-4354-b11a-c0202b10d9b1 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:34.354698 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc75ea-0f39-4354-b11a-c0202b10d9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:34.354974 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc75ea-0f39-4354-b11a-c0202b10d9b1 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:34.355621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc75ea-0f39-4354-b11a-c0202b10d9b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:34.357596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc75ea-0f39-4354-b11a-c0202b10d9b1 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:34.392093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1149/2280] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:54:34 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:54:34.392658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a0f6e-dcef-498b-86b6-4abfbe7647fd admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:34.399067 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a0f6e-dcef-498b-86b6-4abfbe7647fd admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:34.404322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ad94e-bad6-418d-81ac-714e62bf17cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:34.405119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ad94e-bad6-418d-81ac-714e62bf17cd admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:34.405508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ad94e-bad6-418d-81ac-714e62bf17cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:34.405663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ad94e-bad6-418d-81ac-714e62bf17cd admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:34.406299 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ad94e-bad6-418d-81ac-714e62bf17cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:34.407929 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ad94e-bad6-418d-81ac-714e62bf17cd admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:34.418511 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1132/2281] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:54:34.422686 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc75ea-0f39-4354-b11a-c0202b10d9b1 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:34.435140 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc75ea-0f39-4354-b11a-c0202b10d9b1 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:34.440927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa79b7-68c6-4d15-9a09-61af1f3123db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:34.441491 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa79b7-68c6-4d15-9a09-61af1f3123db admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:34.441776 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa79b7-68c6-4d15-9a09-61af1f3123db admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:34.442076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa79b7-68c6-4d15-9a09-61af1f3123db admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:34.442711 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa79b7-68c6-4d15-9a09-61af1f3123db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:34.445182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fa79b7-68c6-4d15-9a09-61af1f3123db admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:34.466729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1150/2282] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:54:34.470463 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ad94e-bad6-418d-81ac-714e62bf17cd admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:34.481050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ad94e-bad6-418d-81ac-714e62bf17cd admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:34.487159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a430a-af88-4119-baf8-bcc936bb04f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:34.488076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a430a-af88-4119-baf8-bcc936bb04f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:34.488464 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a430a-af88-4119-baf8-bcc936bb04f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:34.488913 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a430a-af88-4119-baf8-bcc936bb04f2 admin admin] PATH_INFO: `/v3/projects/aeec00f3a752480cb9ac3a670bcb297d/users/c0b677827f1648319b0616aec4db66e6/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:34.507516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a430a-af88-4119-baf8-bcc936bb04f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeec00f3a752480cb9ac3a670bcb297d, user_id=c0b677827f1648319b0616aec4db66e6, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:34.510309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a430a-af88-4119-baf8-bcc936bb04f2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:34.524764 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1133/2283] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:54:34 2022] PUT /identity/v3/projects/aeec00f3a752480cb9ac3a670bcb297d/users/c0b677827f1648319b0616aec4db66e6/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:54:34.529849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeede57-d43f-4d8d-9459-39f6e855a447 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:34.530188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeede57-d43f-4d8d-9459-39f6e855a447 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:34.530465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeede57-d43f-4d8d-9459-39f6e855a447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:34.550882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-edeede57-d43f-4d8d-9459-39f6e855a447 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:34.552162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-edeede57-d43f-4d8d-9459-39f6e855a447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:34.562402 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:34.624264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1151/2284] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:34.636271 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a430a-af88-4119-baf8-bcc936bb04f2 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:34.638325 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b64a430a-af88-4119-baf8-bcc936bb04f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:34.712410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a430a-af88-4119-baf8-bcc936bb04f2 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:34.716595 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6ef82-6c6d-489e-9893-e005427fb284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:34.717146 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6ef82-6c6d-489e-9893-e005427fb284 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:34.717466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6ef82-6c6d-489e-9893-e005427fb284 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:34.717676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d6ef82-6c6d-489e-9893-e005427fb284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:34.727818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6ef82-6c6d-489e-9893-e005427fb284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:34.729439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d6ef82-6c6d-489e-9893-e005427fb284 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:34.741538 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1134/2285] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:54:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:54:35.545621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd58cb-bd34-4612-82dc-5baf14f85b6d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:35.546148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd58cb-bd34-4612-82dc-5baf14f85b6d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:35.546148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd58cb-bd34-4612-82dc-5baf14f85b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:35.588536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-96cd58cb-bd34-4612-82dc-5baf14f85b6d None None] MFA Rules not processed for user `c0b677827f1648319b0616aec4db66e6`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:35.590528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-96cd58cb-bd34-4612-82dc-5baf14f85b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:35.671431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1152/2286] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:35.683032 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6ef82-6c6d-489e-9893-e005427fb284 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:35.684623 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-30d6ef82-6c6d-489e-9893-e005427fb284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:35.783623 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d6ef82-6c6d-489e-9893-e005427fb284 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:35.788858 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c7c82-e3ad-473b-89ec-73ab234314be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:35.789681 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c7c82-e3ad-473b-89ec-73ab234314be service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:35.789961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c7c82-e3ad-473b-89ec-73ab234314be service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:35.790232 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c7c82-e3ad-473b-89ec-73ab234314be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:35.798853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c7c82-e3ad-473b-89ec-73ab234314be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:35.800692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c7c82-e3ad-473b-89ec-73ab234314be service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:35.811010 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1135/2287] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:54:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:54:36.313156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f64ed-1e6e-489b-b80c-2039eb931d33 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:36.313449 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f64ed-1e6e-489b-b80c-2039eb931d33 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:36.313757 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f64ed-1e6e-489b-b80c-2039eb931d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:36.333335 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-608f64ed-1e6e-489b-b80c-2039eb931d33 None None] MFA Rules not processed for user `61743a63def7451d9b739f3b17afe8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:36.334560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-608f64ed-1e6e-489b-b80c-2039eb931d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:36.390875 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1153/2288] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:37.272482 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196068f-a168-4702-b4d9-f76dc0f630c0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:37.273601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196068f-a168-4702-b4d9-f76dc0f630c0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:37.274061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196068f-a168-4702-b4d9-f76dc0f630c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:37.298753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b196068f-a168-4702-b4d9-f76dc0f630c0 None None] MFA Rules not processed for user `c0b677827f1648319b0616aec4db66e6`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:37.300564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b196068f-a168-4702-b4d9-f76dc0f630c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:37.321921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1136/2289] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:45.370941 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f2959-159e-47ed-9056-86785969e360 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:45.371424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f2959-159e-47ed-9056-86785969e360 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:45.371545 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f2959-159e-47ed-9056-86785969e360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:45.391626 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d66f2959-159e-47ed-9056-86785969e360 None None] MFA Rules not processed for user `1cccf354894c4311b7710d8a65682eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:45.392881 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d66f2959-159e-47ed-9056-86785969e360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:45.442690 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:45.456863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1154/2290] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:54:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:54:45.656994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ef894-78ab-4a10-ba99-faed2663f1ba None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:45.657534 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ef894-78ab-4a10-ba99-faed2663f1ba None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:45.657534 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ef894-78ab-4a10-ba99-faed2663f1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:45.677407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ce4ef894-78ab-4a10-ba99-faed2663f1ba None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:54:45.678665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ce4ef894-78ab-4a10-ba99-faed2663f1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:45.683962 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:45.742447 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1137/2291] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Sep 19 14:54:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:54:45.755760 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f2959-159e-47ed-9056-86785969e360 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:54:45.757088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d66f2959-159e-47ed-9056-86785969e360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:54:45.833314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f2959-159e-47ed-9056-86785969e360 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:54:45.838469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58ef59-8b68-41e4-b295-944b65e47c29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:54:45.839105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58ef59-8b68-41e4-b295-944b65e47c29 service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:54:45.839391 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58ef59-8b68-41e4-b295-944b65e47c29 service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:54:45.839657 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58ef59-8b68-41e4-b295-944b65e47c29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:54:45.848031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58ef59-8b68-41e4-b295-944b65e47c29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:54:45.849602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58ef59-8b68-41e4-b295-944b65e47c29 service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:54:45.857956 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1155/2292] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 14:54:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 14:54:55.735072 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=21 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:54:55.884301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=35 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:55:02.900904 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cca6c5-a188-4198-8bfc-1d203422d327 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:02.901710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cca6c5-a188-4198-8bfc-1d203422d327 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:02.901710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cca6c5-a188-4198-8bfc-1d203422d327 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:02.903357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1138/2293] 10.222.0.60 () {54 vars in 921 bytes} [Mon Sep 19 14:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:55:02.909485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa6869-d2ff-48cf-81ae-3aff957f45f1 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:02.910153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa6869-d2ff-48cf-81ae-3aff957f45f1 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:02.910153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa6869-d2ff-48cf-81ae-3aff957f45f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:02.932184 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-52aa6869-d2ff-48cf-81ae-3aff957f45f1 None None] MFA Rules not processed for user `59b5d2fd362b417c9b9be75f88a667ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:55:02.933531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-52aa6869-d2ff-48cf-81ae-3aff957f45f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:03.013065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1156/2294] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Sep 19 14:55:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:55:03.025370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cca6c5-a188-4198-8bfc-1d203422d327 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:03.027208 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-76cca6c5-a188-4198-8bfc-1d203422d327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:03.119868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cca6c5-a188-4198-8bfc-1d203422d327 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:03.123905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811020d-2a4a-44cc-a22c-4062035e32b4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caab7aaa72634b358d717768fbcb48d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:03.124516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811020d-2a4a-44cc-a22c-4062035e32b4 service glance] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:03.124804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811020d-2a4a-44cc-a22c-4062035e32b4 service glance] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:03.125076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811020d-2a4a-44cc-a22c-4062035e32b4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:03.140410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811020d-2a4a-44cc-a22c-4062035e32b4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:03.141994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811020d-2a4a-44cc-a22c-4062035e32b4 service glance] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:03.154770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1139/2295] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:55:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:55:03.227642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe1967-79c5-47fa-aeaf-868b18a8fc7a None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:03.228024 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe1967-79c5-47fa-aeaf-868b18a8fc7a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:03.228278 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe1967-79c5-47fa-aeaf-868b18a8fc7a None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:03.229968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1157/2296] 10.222.0.60 () {54 vars in 921 bytes} [Mon Sep 19 14:55:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:55:03.234694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b56aa7-f2b9-465a-81f1-63b3c2e784fb None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:03.234988 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b56aa7-f2b9-465a-81f1-63b3c2e784fb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:03.235268 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b56aa7-f2b9-465a-81f1-63b3c2e784fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:03.258954 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d4b56aa7-f2b9-465a-81f1-63b3c2e784fb None None] MFA Rules not processed for user `2fbef7b24f134f5eb1193aed1b7ee69c`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:55:03.260615 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d4b56aa7-f2b9-465a-81f1-63b3c2e784fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:03.267459 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1140/2297] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Sep 19 14:55:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:55:03.279913 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe1967-79c5-47fa-aeaf-868b18a8fc7a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:03.281672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-92fe1967-79c5-47fa-aeaf-868b18a8fc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:03.296743 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe1967-79c5-47fa-aeaf-868b18a8fc7a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:03.300477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a873b1-e8e5-4a73-915e-bd066df9f81e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:03.301088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a873b1-e8e5-4a73-915e-bd066df9f81e service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:03.301357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a873b1-e8e5-4a73-915e-bd066df9f81e service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:03.301624 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a873b1-e8e5-4a73-915e-bd066df9f81e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:03.311846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a873b1-e8e5-4a73-915e-bd066df9f81e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:03.313400 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a873b1-e8e5-4a73-915e-bd066df9f81e service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:03.323264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1158/2298] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:55:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:55:03.433909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b6602-a0aa-4fee-9963-2ebb7b4ab932 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:03.434198 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b6602-a0aa-4fee-9963-2ebb7b4ab932 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:03.434474 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b6602-a0aa-4fee-9963-2ebb7b4ab932 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:03.435894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1141/2299] 10.222.0.60 () {54 vars in 921 bytes} [Mon Sep 19 14:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:55:03.446859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67f3b6-a3b3-427f-883f-df3c65c2aaac None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:03.447729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67f3b6-a3b3-427f-883f-df3c65c2aaac None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:03.452397 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67f3b6-a3b3-427f-883f-df3c65c2aaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:03.472158 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-4c67f3b6-a3b3-427f-883f-df3c65c2aaac None None] MFA Rules not processed for user `59b5d2fd362b417c9b9be75f88a667ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:55:03.473464 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4c67f3b6-a3b3-427f-883f-df3c65c2aaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:03.483154 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1159/2300] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Sep 19 14:55:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:55:03.494345 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b6602-a0aa-4fee-9963-2ebb7b4ab932 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:03.505255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b6602-a0aa-4fee-9963-2ebb7b4ab932 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:03.509489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33591693-3609-44ac-98db-c5cc7f9f504a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:03.510139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33591693-3609-44ac-98db-c5cc7f9f504a service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:03.510530 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33591693-3609-44ac-98db-c5cc7f9f504a service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:03.510676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33591693-3609-44ac-98db-c5cc7f9f504a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:03.521136 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33591693-3609-44ac-98db-c5cc7f9f504a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:03.522619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33591693-3609-44ac-98db-c5cc7f9f504a service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:03.535858 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1142/2301] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:55:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:55:06.442303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248e561-12a3-47da-a034-a6e050319066 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:06.442784 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248e561-12a3-47da-a034-a6e050319066 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:06.442882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248e561-12a3-47da-a034-a6e050319066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:06.463527 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0248e561-12a3-47da-a034-a6e050319066 None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:55:06.464965 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0248e561-12a3-47da-a034-a6e050319066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:06.473790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1160/2302] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Sep 19 14:55:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:55:06.488279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33591693-3609-44ac-98db-c5cc7f9f504a service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:06.507180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33591693-3609-44ac-98db-c5cc7f9f504a service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:06.515694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb69bd4-e87d-4420-9b09-beac4b5f515a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:06.516358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb69bd4-e87d-4420-9b09-beac4b5f515a service swift] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:06.516639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb69bd4-e87d-4420-9b09-beac4b5f515a service swift] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:06.516909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb69bd4-e87d-4420-9b09-beac4b5f515a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:06.526645 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb69bd4-e87d-4420-9b09-beac4b5f515a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:06.528289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb69bd4-e87d-4420-9b09-beac4b5f515a service swift] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:06.538223 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1143/2303] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 14:55:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 14:55:16.457076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=43 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:55:16.534392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=50 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:55:26.280196 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248e561-12a3-47da-a034-a6e050319066 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:26.281901 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0248e561-12a3-47da-a034-a6e050319066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:26.321859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248e561-12a3-47da-a034-a6e050319066 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:26.333520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f098f1f-0307-4d8e-8f1c-765895fbcd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:26.334441 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f098f1f-0307-4d8e-8f1c-765895fbcd81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:26.335489 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f098f1f-0307-4d8e-8f1c-765895fbcd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:26.337021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f098f1f-0307-4d8e-8f1c-765895fbcd81 admin admin] PATH_INFO: `/v3/users/218c699b6ddd43a4bcf9b842c59445f5` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:26.338469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f098f1f-0307-4d8e-8f1c-765895fbcd81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=218c699b6ddd43a4bcf9b842c59445f5)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:26.340612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f098f1f-0307-4d8e-8f1c-765895fbcd81 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:26.419292 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7f098f1f-0307-4d8e-8f1c-765895fbcd81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '218c699b6ddd43a4bcf9b842c59445f5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:26.430541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7f098f1f-0307-4d8e-8f1c-765895fbcd81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '218c699b6ddd43a4bcf9b842c59445f5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:26.446148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7f098f1f-0307-4d8e-8f1c-765895fbcd81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '218c699b6ddd43a4bcf9b842c59445f5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:26.461995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7f098f1f-0307-4d8e-8f1c-765895fbcd81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '218c699b6ddd43a4bcf9b842c59445f5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:26.462777 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7f098f1f-0307-4d8e-8f1c-765895fbcd81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '218c699b6ddd43a4bcf9b842c59445f5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:26.470321 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1161/2304] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:55:26 2022] DELETE /identity/v3/users/218c699b6ddd43a4bcf9b842c59445f5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:55:26.663681 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb69bd4-e87d-4420-9b09-beac4b5f515a service swift] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:26.677090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb69bd4-e87d-4420-9b09-beac4b5f515a service swift] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:26.738259 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef663293-7f38-48af-bea8-4e090d96fd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:26.738855 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef663293-7f38-48af-bea8-4e090d96fd7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:26.739139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef663293-7f38-48af-bea8-4e090d96fd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:26.739410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef663293-7f38-48af-bea8-4e090d96fd7c admin admin] PATH_INFO: `/v3/projects/5a6e2209a3a3487a9d6a7255cbe56b63` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:26.740556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef663293-7f38-48af-bea8-4e090d96fd7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a6e2209a3a3487a9d6a7255cbe56b63)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:26.742517 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef663293-7f38-48af-bea8-4e090d96fd7c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:26.788731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ef663293-7f38-48af-bea8-4e090d96fd7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a6e2209a3a3487a9d6a7255cbe56b63', 'request_id': 'req-ef663293-7f38-48af-bea8-4e090d96fd7c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:26.794669 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ef663293-7f38-48af-bea8-4e090d96fd7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a6e2209a3a3487a9d6a7255cbe56b63', 'request_id': 'req-ef663293-7f38-48af-bea8-4e090d96fd7c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:26.802319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ef663293-7f38-48af-bea8-4e090d96fd7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a6e2209a3a3487a9d6a7255cbe56b63', 'request_id': 'req-ef663293-7f38-48af-bea8-4e090d96fd7c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:26.813100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ef663293-7f38-48af-bea8-4e090d96fd7c admin admin] The token cache is being invalidate because project 5a6e2209a3a3487a9d6a7255cbe56b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:55:26.813396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ef663293-7f38-48af-bea8-4e090d96fd7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:26.814038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ef663293-7f38-48af-bea8-4e090d96fd7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:26.815530 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1144/2305] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:55:26 2022] DELETE /identity/v3/projects/5a6e2209a3a3487a9d6a7255cbe56b63 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:55:26.819504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f098f1f-0307-4d8e-8f1c-765895fbcd81 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:26.821505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7f098f1f-0307-4d8e-8f1c-765895fbcd81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:26.834624 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f098f1f-0307-4d8e-8f1c-765895fbcd81 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:26.903238 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0b7fb-af1f-422f-83e3-d0ff814d2499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:26.904051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0b7fb-af1f-422f-83e3-d0ff814d2499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:26.904367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0b7fb-af1f-422f-83e3-d0ff814d2499 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:26.904642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0b7fb-af1f-422f-83e3-d0ff814d2499 admin admin] PATH_INFO: `/v3/users/05f6ed5c0d6a45419f3add7085bd24c8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:26.905880 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0b7fb-af1f-422f-83e3-d0ff814d2499 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05f6ed5c0d6a45419f3add7085bd24c8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:26.907806 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0b7fb-af1f-422f-83e3-d0ff814d2499 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:26.985245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c3a0b7fb-af1f-422f-83e3-d0ff814d2499 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05f6ed5c0d6a45419f3add7085bd24c8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:26.994155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c3a0b7fb-af1f-422f-83e3-d0ff814d2499 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05f6ed5c0d6a45419f3add7085bd24c8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:27.012919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c3a0b7fb-af1f-422f-83e3-d0ff814d2499 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05f6ed5c0d6a45419f3add7085bd24c8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:27.014590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:55:27.023697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c3a0b7fb-af1f-422f-83e3-d0ff814d2499 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05f6ed5c0d6a45419f3add7085bd24c8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:27.024873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c3a0b7fb-af1f-422f-83e3-d0ff814d2499 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05f6ed5c0d6a45419f3add7085bd24c8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:27.032826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1162/2306] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:55:26 2022] DELETE /identity/v3/users/05f6ed5c0d6a45419f3add7085bd24c8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:55:27.231900 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef663293-7f38-48af-bea8-4e090d96fd7c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:27.234470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ef663293-7f38-48af-bea8-4e090d96fd7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:27.249953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef663293-7f38-48af-bea8-4e090d96fd7c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:27.307352 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a089b8-de59-4ee9-b10c-f44ef609df84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:27.307897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a089b8-de59-4ee9-b10c-f44ef609df84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:27.308177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a089b8-de59-4ee9-b10c-f44ef609df84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:27.308471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a089b8-de59-4ee9-b10c-f44ef609df84 admin admin] PATH_INFO: `/v3/projects/186f31f2145548c3b1097435da6d306e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:27.309582 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a089b8-de59-4ee9-b10c-f44ef609df84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=186f31f2145548c3b1097435da6d306e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:27.311471 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a089b8-de59-4ee9-b10c-f44ef609df84 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:27.346214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:55:27.361126 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-24a089b8-de59-4ee9-b10c-f44ef609df84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '186f31f2145548c3b1097435da6d306e', 'request_id': 'req-24a089b8-de59-4ee9-b10c-f44ef609df84'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:27.367521 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-24a089b8-de59-4ee9-b10c-f44ef609df84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '186f31f2145548c3b1097435da6d306e', 'request_id': 'req-24a089b8-de59-4ee9-b10c-f44ef609df84'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:27.375074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-24a089b8-de59-4ee9-b10c-f44ef609df84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '186f31f2145548c3b1097435da6d306e', 'request_id': 'req-24a089b8-de59-4ee9-b10c-f44ef609df84'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:27.387566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-24a089b8-de59-4ee9-b10c-f44ef609df84 admin admin] The token cache is being invalidate because project 186f31f2145548c3b1097435da6d306e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:55:27.387953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-24a089b8-de59-4ee9-b10c-f44ef609df84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:27.388824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-24a089b8-de59-4ee9-b10c-f44ef609df84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:55:27.390408 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1145/2307] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:55:27 2022] DELETE /identity/v3/projects/186f31f2145548c3b1097435da6d306e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:55:27.395333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae808ea-8729-4ed0-94bf-e5355dcaf206 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:27.395809 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae808ea-8729-4ed0-94bf-e5355dcaf206 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:27.395939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae808ea-8729-4ed0-94bf-e5355dcaf206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:27.416371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-fae808ea-8729-4ed0-94bf-e5355dcaf206 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:55:27.417948 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fae808ea-8729-4ed0-94bf-e5355dcaf206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:27.480573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1163/2308] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:55:27.489517 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989fcf2-73c2-4a22-ade6-9e8e3c62238b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:27.489806 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989fcf2-73c2-4a22-ade6-9e8e3c62238b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:27.490084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989fcf2-73c2-4a22-ade6-9e8e3c62238b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:27.510367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-4989fcf2-73c2-4a22-ade6-9e8e3c62238b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:55:27.511557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4989fcf2-73c2-4a22-ade6-9e8e3c62238b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:27.523140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1146/2309] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:55:27.524801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae808ea-8729-4ed0-94bf-e5355dcaf206 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:27.534615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae808ea-8729-4ed0-94bf-e5355dcaf206 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:27.539268 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf4dac-23d1-462f-9e5f-d1c171db7923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:27.539917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf4dac-23d1-462f-9e5f-d1c171db7923 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:27.540176 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf4dac-23d1-462f-9e5f-d1c171db7923 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:27.540458 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf4dac-23d1-462f-9e5f-d1c171db7923 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:27.541091 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf4dac-23d1-462f-9e5f-d1c171db7923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:55:27.541390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf4dac-23d1-462f-9e5f-d1c171db7923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:27.542967 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bf4dac-23d1-462f-9e5f-d1c171db7923 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:27.555349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1164/2310] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:55:27.560226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989fcf2-73c2-4a22-ade6-9e8e3c62238b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:27.573021 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989fcf2-73c2-4a22-ade6-9e8e3c62238b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:27.577718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3736aa-1e51-4db2-9261-8df06d3d380d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:27.578378 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3736aa-1e51-4db2-9261-8df06d3d380d admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:27.578686 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3736aa-1e51-4db2-9261-8df06d3d380d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:27.579029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3736aa-1e51-4db2-9261-8df06d3d380d admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:27.579758 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3736aa-1e51-4db2-9261-8df06d3d380d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:27.581822 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3736aa-1e51-4db2-9261-8df06d3d380d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:27.613682 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1147/2311] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:55:27 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:55:27.617957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf4dac-23d1-462f-9e5f-d1c171db7923 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:27.628609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bf4dac-23d1-462f-9e5f-d1c171db7923 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:27.686998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce9bc3-746e-46dc-9b74-5b0f290294d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:27.687609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9bc3-746e-46dc-9b74-5b0f290294d3 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:27.687915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9bc3-746e-46dc-9b74-5b0f290294d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:27.688188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce9bc3-746e-46dc-9b74-5b0f290294d3 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:27.688818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce9bc3-746e-46dc-9b74-5b0f290294d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:27.690700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce9bc3-746e-46dc-9b74-5b0f290294d3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:27.712733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1165/2312] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:55:27 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:55:27.717163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3736aa-1e51-4db2-9261-8df06d3d380d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:27.727413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3736aa-1e51-4db2-9261-8df06d3d380d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:27.732203 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a25fa3-71e4-47e2-a23d-9ff256e07c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:27.732834 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a25fa3-71e4-47e2-a23d-9ff256e07c21 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:27.733108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a25fa3-71e4-47e2-a23d-9ff256e07c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:27.733377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a25fa3-71e4-47e2-a23d-9ff256e07c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:27.734034 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a25fa3-71e4-47e2-a23d-9ff256e07c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:27.735626 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a25fa3-71e4-47e2-a23d-9ff256e07c21 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:27.745979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1148/2313] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:55:27.749986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce9bc3-746e-46dc-9b74-5b0f290294d3 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:27.760807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce9bc3-746e-46dc-9b74-5b0f290294d3 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:27.765626 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d47417-aa1f-40b3-8155-a19183521b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:27.766237 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d47417-aa1f-40b3-8155-a19183521b81 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:27.767199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d47417-aa1f-40b3-8155-a19183521b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:27.767476 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d47417-aa1f-40b3-8155-a19183521b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:27.768215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d47417-aa1f-40b3-8155-a19183521b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:27.770103 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d47417-aa1f-40b3-8155-a19183521b81 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:27.781707 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1166/2314] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:55:27.786156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a25fa3-71e4-47e2-a23d-9ff256e07c21 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:27.796882 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a25fa3-71e4-47e2-a23d-9ff256e07c21 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:27.801879 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382f25d-0a86-4d57-9a75-0e9ea81d6e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:27.802574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382f25d-0a86-4d57-9a75-0e9ea81d6e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:27.802861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382f25d-0a86-4d57-9a75-0e9ea81d6e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:27.803140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382f25d-0a86-4d57-9a75-0e9ea81d6e57 admin admin] PATH_INFO: `/v3/projects/cbb13ea6de2a48e99ce93be06fd87830/users/52173f24f42a481380b047abe76bca57/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:27.822708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382f25d-0a86-4d57-9a75-0e9ea81d6e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb13ea6de2a48e99ce93be06fd87830, user_id=52173f24f42a481380b047abe76bca57, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:27.825439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382f25d-0a86-4d57-9a75-0e9ea81d6e57 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:27.840428 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1149/2315] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:55:27 2022] PUT /identity/v3/projects/cbb13ea6de2a48e99ce93be06fd87830/users/52173f24f42a481380b047abe76bca57/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:55:27.847631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a5438-567c-458a-ac38-a969b9c7813c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:27.848061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a5438-567c-458a-ac38-a969b9c7813c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:27.848407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a5438-567c-458a-ac38-a969b9c7813c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:27.869969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-838a5438-567c-458a-ac38-a969b9c7813c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:55:27.871370 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-838a5438-567c-458a-ac38-a969b9c7813c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:27.930841 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:55:27.942800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1167/2316] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:55:27.953143 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382f25d-0a86-4d57-9a75-0e9ea81d6e57 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:27.954841 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1382f25d-0a86-4d57-9a75-0e9ea81d6e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:28.011445 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382f25d-0a86-4d57-9a75-0e9ea81d6e57 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:28.015099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245aca8e-09e2-474f-9cf4-829adf6bca21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:28.015697 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245aca8e-09e2-474f-9cf4-829adf6bca21 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:28.016004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245aca8e-09e2-474f-9cf4-829adf6bca21 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:28.016278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245aca8e-09e2-474f-9cf4-829adf6bca21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:28.025281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245aca8e-09e2-474f-9cf4-829adf6bca21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:28.026699 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245aca8e-09e2-474f-9cf4-829adf6bca21 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:28.036053 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1150/2317] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:55:32.760088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd37e39-b39d-404f-9cc3-0bb7d2bf897f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:32.760583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd37e39-b39d-404f-9cc3-0bb7d2bf897f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:32.760668 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd37e39-b39d-404f-9cc3-0bb7d2bf897f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:32.792802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8fd37e39-b39d-404f-9cc3-0bb7d2bf897f None None] MFA Rules not processed for user `52173f24f42a481380b047abe76bca57`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:55:32.794108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8fd37e39-b39d-404f-9cc3-0bb7d2bf897f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:32.878371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1168/2318] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:55:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:55:32.882843 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245aca8e-09e2-474f-9cf4-829adf6bca21 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:32.893909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245aca8e-09e2-474f-9cf4-829adf6bca21 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:32.899705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed7e0c-3933-4753-b3c3-c3ed2428d305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:32.900377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed7e0c-3933-4753-b3c3-c3ed2428d305 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:32.900662 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed7e0c-3933-4753-b3c3-c3ed2428d305 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:32.900937 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed7e0c-3933-4753-b3c3-c3ed2428d305 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:32.901574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed7e0c-3933-4753-b3c3-c3ed2428d305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:32.903518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed7e0c-3933-4753-b3c3-c3ed2428d305 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:32.948203 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1151/2319] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:55:32 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:55:32.950056 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd37e39-b39d-404f-9cc3-0bb7d2bf897f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:32.962832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd37e39-b39d-404f-9cc3-0bb7d2bf897f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:33.030742 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d878bb-cfdf-44e6-8a0f-bea353694739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:33.031525 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d878bb-cfdf-44e6-8a0f-bea353694739 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:33.031941 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d878bb-cfdf-44e6-8a0f-bea353694739 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:33.032230 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d878bb-cfdf-44e6-8a0f-bea353694739 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:33.032887 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d878bb-cfdf-44e6-8a0f-bea353694739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:33.034951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d878bb-cfdf-44e6-8a0f-bea353694739 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:33.059324 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1169/2320] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:55:32 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:55:33.063924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed7e0c-3933-4753-b3c3-c3ed2428d305 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:33.076460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed7e0c-3933-4753-b3c3-c3ed2428d305 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:33.082447 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634d6c3-6eee-4bbc-9132-b6d11a1f103f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:33.083157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634d6c3-6eee-4bbc-9132-b6d11a1f103f admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:33.083526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634d6c3-6eee-4bbc-9132-b6d11a1f103f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:33.083899 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634d6c3-6eee-4bbc-9132-b6d11a1f103f admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:33.084631 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634d6c3-6eee-4bbc-9132-b6d11a1f103f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:33.087045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634d6c3-6eee-4bbc-9132-b6d11a1f103f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:33.102001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1152/2321] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:55:33.106643 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d878bb-cfdf-44e6-8a0f-bea353694739 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:33.117647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d878bb-cfdf-44e6-8a0f-bea353694739 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:33.124416 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e4126-fd9b-4e8b-94f7-f466cf44e30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:33.125376 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e4126-fd9b-4e8b-94f7-f466cf44e30a admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:33.125912 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e4126-fd9b-4e8b-94f7-f466cf44e30a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:33.126378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e4126-fd9b-4e8b-94f7-f466cf44e30a admin admin] PATH_INFO: `/v3/projects/6b7766406094495b9d7eb12fcc35aee3/users/ba481d9001c54a11946049a06185790a/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:33.144483 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e4126-fd9b-4e8b-94f7-f466cf44e30a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b7766406094495b9d7eb12fcc35aee3, user_id=ba481d9001c54a11946049a06185790a, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:33.147523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e4126-fd9b-4e8b-94f7-f466cf44e30a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:33.168249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1170/2322] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:55:33 2022] PUT /identity/v3/projects/6b7766406094495b9d7eb12fcc35aee3/users/ba481d9001c54a11946049a06185790a/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:55:33.172206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634d6c3-6eee-4bbc-9132-b6d11a1f103f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:33.184578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634d6c3-6eee-4bbc-9132-b6d11a1f103f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:33.212516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:55:33.254493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0d413-a44d-4e7a-b5e1-bd5da641eadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:33.255627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0d413-a44d-4e7a-b5e1-bd5da641eadb admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:33.256004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0d413-a44d-4e7a-b5e1-bd5da641eadb admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:33.256343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0d413-a44d-4e7a-b5e1-bd5da641eadb admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:33.257064 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0d413-a44d-4e7a-b5e1-bd5da641eadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:33.259429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0d413-a44d-4e7a-b5e1-bd5da641eadb admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:33.276807 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1153/2323] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:55:33.281125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e4126-fd9b-4e8b-94f7-f466cf44e30a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:33.295085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e4126-fd9b-4e8b-94f7-f466cf44e30a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:33.301308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f520e3a9-9413-412f-babb-4e5454227664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:33.302096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520e3a9-9413-412f-babb-4e5454227664 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:33.302500 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520e3a9-9413-412f-babb-4e5454227664 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:33.302914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520e3a9-9413-412f-babb-4e5454227664 admin admin] PATH_INFO: `/v3/projects/6b7766406094495b9d7eb12fcc35aee3/users/ba481d9001c54a11946049a06185790a/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:33.308160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520e3a9-9413-412f-babb-4e5454227664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b7766406094495b9d7eb12fcc35aee3, user_id=ba481d9001c54a11946049a06185790a, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:33.312068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f520e3a9-9413-412f-babb-4e5454227664 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:33.332252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1171/2324] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:55:33 2022] PUT /identity/v3/projects/6b7766406094495b9d7eb12fcc35aee3/users/ba481d9001c54a11946049a06185790a/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:55:35.295960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e14db-3814-4977-8e0d-ed5d34d4c727 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:35.296916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e14db-3814-4977-8e0d-ed5d34d4c727 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:35.297479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e14db-3814-4977-8e0d-ed5d34d4c727 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:35.300080 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1154/2325] 10.222.0.60 () {54 vars in 961 bytes} [Mon Sep 19 14:55:35 2022] GET /identity => generated 270 bytes in 28 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:55:35.315004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c02e16-8209-456d-85e9-fecbaf7afad2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:35.315604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c02e16-8209-456d-85e9-fecbaf7afad2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:35.316041 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c02e16-8209-456d-85e9-fecbaf7afad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:35.351660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-55c02e16-8209-456d-85e9-fecbaf7afad2 None None] MFA Rules not processed for user `568189358b154b66a729fae2f0411d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:55:35.355377 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-55c02e16-8209-456d-85e9-fecbaf7afad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:35.470961 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:55:35.543583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1172/2326] 10.222.0.60 () {58 vars in 1058 bytes} [Mon Sep 19 14:55:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:55:35.559870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e14db-3814-4977-8e0d-ed5d34d4c727 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:35.592513 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e14db-3814-4977-8e0d-ed5d34d4c727 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:35.598106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b7935-50fa-4bf8-a611-26342c18091b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '568189358b154b66a729fae2f0411d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:35.598727 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b7935-50fa-4bf8-a611-26342c18091b service placement] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:35.599286 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b7935-50fa-4bf8-a611-26342c18091b service placement] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:35.599710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b7935-50fa-4bf8-a611-26342c18091b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:35.613522 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0c6b7935-50fa-4bf8-a611-26342c18091b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:35.640175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b7935-50fa-4bf8-a611-26342c18091b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:35.642030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b7935-50fa-4bf8-a611-26342c18091b service placement] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:35.662332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1155/2327] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:55:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:55:38.472500 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2968b-6fba-4403-a3d3-bf23d6dd54a7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:38.473014 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2968b-6fba-4403-a3d3-bf23d6dd54a7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:38.473144 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e2968b-6fba-4403-a3d3-bf23d6dd54a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:38.503173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-23e2968b-6fba-4403-a3d3-bf23d6dd54a7 None None] MFA Rules not processed for user `ba481d9001c54a11946049a06185790a`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:55:38.504494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-23e2968b-6fba-4403-a3d3-bf23d6dd54a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:38.581488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1173/2328] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:55:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:55:38.595750 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b7935-50fa-4bf8-a611-26342c18091b service placement] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:38.597081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0c6b7935-50fa-4bf8-a611-26342c18091b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:38.679643 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b7935-50fa-4bf8-a611-26342c18091b service placement] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:38.685983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0371c-a5d9-4769-b0cd-f75c73830b35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:38.686594 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0371c-a5d9-4769-b0cd-f75c73830b35 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:38.686875 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0371c-a5d9-4769-b0cd-f75c73830b35 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:38.687148 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0371c-a5d9-4769-b0cd-f75c73830b35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:38.761301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0371c-a5d9-4769-b0cd-f75c73830b35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:38.762899 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0371c-a5d9-4769-b0cd-f75c73830b35 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:38.772432 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1156/2329] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:55:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:55:41.924017 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55fa04-2f7e-4cfc-a5dd-0c8dcd63bb5b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:41.924508 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55fa04-2f7e-4cfc-a5dd-0c8dcd63bb5b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:41.924606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55fa04-2f7e-4cfc-a5dd-0c8dcd63bb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:41.945147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3c55fa04-2f7e-4cfc-a5dd-0c8dcd63bb5b None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:55:41.946466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3c55fa04-2f7e-4cfc-a5dd-0c8dcd63bb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:42.028641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1174/2330] 10.222.0.60 () {58 vars in 967 bytes} [Mon Sep 19 14:55:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:55:42.055874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0371c-a5d9-4769-b0cd-f75c73830b35 service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:55:42.058316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f0c0371c-a5d9-4769-b0cd-f75c73830b35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:42.127630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:55:42.152613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0371c-a5d9-4769-b0cd-f75c73830b35 service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:55:42.160743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb628c-02ca-4395-bd5e-0268b891fd59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:55:42.161639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb628c-02ca-4395-bd5e-0268b891fd59 service swift] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:42.162163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb628c-02ca-4395-bd5e-0268b891fd59 service swift] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:42.162593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb628c-02ca-4395-bd5e-0268b891fd59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:42.172621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb628c-02ca-4395-bd5e-0268b891fd59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:55:42.174722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbb628c-02ca-4395-bd5e-0268b891fd59 service swift] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:55:42.185059 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1157/2331] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 14:55:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 14:55:52.015895 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=70 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:55:52.182654 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=11 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:55:58.877507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89031b-6eb0-47da-8cac-3e4e192b8a98 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:58.878180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89031b-6eb0-47da-8cac-3e4e192b8a98 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:58.878180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89031b-6eb0-47da-8cac-3e4e192b8a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:55:58.898220 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2a89031b-6eb0-47da-8cac-3e4e192b8a98 None None] MFA Rules not processed for user `c0b677827f1648319b0616aec4db66e6`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:55:58.900453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2a89031b-6eb0-47da-8cac-3e4e192b8a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:55:58.961781 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1175/2332] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:55:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:55:59.994635 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90796234-5816-473f-9e73-1566372ccc21 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:55:59.995175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90796234-5816-473f-9e73-1566372ccc21 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:55:59.995299 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90796234-5816-473f-9e73-1566372ccc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:56:00.015158 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-90796234-5816-473f-9e73-1566372ccc21 None None] MFA Rules not processed for user `52173f24f42a481380b047abe76bca57`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:56:00.016379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-90796234-5816-473f-9e73-1566372ccc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:56:00.023353 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1158/2333] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:55:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:56:08.954435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=25 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:56:10.009433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:56:19.671674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0089c0c-5944-4155-8021-c541aadb3bd0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:56:19.672314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0089c0c-5944-4155-8021-c541aadb3bd0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:56:19.672314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0089c0c-5944-4155-8021-c541aadb3bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:56:19.692803 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b0089c0c-5944-4155-8021-c541aadb3bd0 None None] MFA Rules not processed for user `61743a63def7451d9b739f3b17afe8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:56:19.694065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b0089c0c-5944-4155-8021-c541aadb3bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:56:19.749509 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1176/2334] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:56:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:56:20.951690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90796234-5816-473f-9e73-1566372ccc21 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:56:21.009882 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90796234-5816-473f-9e73-1566372ccc21 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:56:21.014848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4401bad-0750-4d55-bb29-7feca1afcda7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:56:21.015444 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4401bad-0750-4d55-bb29-7feca1afcda7 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:56:21.015726 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4401bad-0750-4d55-bb29-7feca1afcda7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:56:21.016034 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4401bad-0750-4d55-bb29-7feca1afcda7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:56:21.026788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4401bad-0750-4d55-bb29-7feca1afcda7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:56:21.028305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4401bad-0750-4d55-bb29-7feca1afcda7 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:56:21.040100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1159/2335] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:56:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:56:29.742768 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=25 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:56:31.033872 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=26 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:56:36.685502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc9ff1-f2ea-4b9d-8b1e-ef68c40f4f85 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:56:36.686040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc9ff1-f2ea-4b9d-8b1e-ef68c40f4f85 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:56:36.686040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc9ff1-f2ea-4b9d-8b1e-ef68c40f4f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:56:36.705548 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d2dc9ff1-f2ea-4b9d-8b1e-ef68c40f4f85 None None] MFA Rules not processed for user `1cccf354894c4311b7710d8a65682eeb`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:56:36.706875 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d2dc9ff1-f2ea-4b9d-8b1e-ef68c40f4f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:56:36.764055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1177/2336] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:56:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:56:46.756886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=25 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:56:53.057316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75f9e3-e3dc-419f-aabb-b4182a019bfb None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:56:53.057874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75f9e3-e3dc-419f-aabb-b4182a019bfb None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:56:53.057874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd75f9e3-e3dc-419f-aabb-b4182a019bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:56:53.077046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-bd75f9e3-e3dc-419f-aabb-b4182a019bfb None None] MFA Rules not processed for user `b2c40600d2fd4eea8e4ba999050abd73`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:56:53.078266 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bd75f9e3-e3dc-419f-aabb-b4182a019bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:56:53.134210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1160/2337] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:56:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:57:03.126987 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=25 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:57:12.213566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc9ff1-f2ea-4b9d-8b1e-ef68c40f4f85 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:12.215249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d2dc9ff1-f2ea-4b9d-8b1e-ef68c40f4f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:12.282929 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc9ff1-f2ea-4b9d-8b1e-ef68c40f4f85 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:12.288051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10331fb-5653-492b-8210-39de2278b48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:12.288674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10331fb-5653-492b-8210-39de2278b48d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:12.288962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10331fb-5653-492b-8210-39de2278b48d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:12.289235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10331fb-5653-492b-8210-39de2278b48d admin admin] PATH_INFO: `/v3/users/c0b677827f1648319b0616aec4db66e6` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:12.290358 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10331fb-5653-492b-8210-39de2278b48d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0b677827f1648319b0616aec4db66e6)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:12.292309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10331fb-5653-492b-8210-39de2278b48d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:12.337556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b10331fb-5653-492b-8210-39de2278b48d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0b677827f1648319b0616aec4db66e6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:12.344415 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b10331fb-5653-492b-8210-39de2278b48d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0b677827f1648319b0616aec4db66e6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:12.356098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b10331fb-5653-492b-8210-39de2278b48d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0b677827f1648319b0616aec4db66e6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:12.366615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b10331fb-5653-492b-8210-39de2278b48d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0b677827f1648319b0616aec4db66e6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:12.367218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-b10331fb-5653-492b-8210-39de2278b48d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0b677827f1648319b0616aec4db66e6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:12.373697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1178/2338] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:57:12 2022] DELETE /identity/v3/users/c0b677827f1648319b0616aec4db66e6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:57:12.549906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75f9e3-e3dc-419f-aabb-b4182a019bfb None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:12.562490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd75f9e3-e3dc-419f-aabb-b4182a019bfb None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:12.626580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:12.627212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:12.627496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:12.627770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f admin admin] PATH_INFO: `/v3/projects/aeec00f3a752480cb9ac3a670bcb297d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:12.628946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeec00f3a752480cb9ac3a670bcb297d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:12.631388 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:12.677950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeec00f3a752480cb9ac3a670bcb297d', 'request_id': 'req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:12.684332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeec00f3a752480cb9ac3a670bcb297d', 'request_id': 'req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:12.691694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeec00f3a752480cb9ac3a670bcb297d', 'request_id': 'req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:12.702841 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f admin admin] The token cache is being invalidate because project aeec00f3a752480cb9ac3a670bcb297d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:57:12.703130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:12.703769 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:12.705283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1161/2339] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:57:12 2022] DELETE /identity/v3/projects/aeec00f3a752480cb9ac3a670bcb297d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:57:20.865151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10331fb-5653-492b-8210-39de2278b48d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:20.866582 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b10331fb-5653-492b-8210-39de2278b48d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:20.924257 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10331fb-5653-492b-8210-39de2278b48d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:20.957047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:57:20.978103 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb669e-50b8-4ab2-93b2-3327a0bd38f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:20.978724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb669e-50b8-4ab2-93b2-3327a0bd38f0 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:20.979030 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb669e-50b8-4ab2-93b2-3327a0bd38f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:20.979306 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb669e-50b8-4ab2-93b2-3327a0bd38f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:20.981136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c2bb669e-50b8-4ab2-93b2-3327a0bd38f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:21.058007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb669e-50b8-4ab2-93b2-3327a0bd38f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:21.059450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb669e-50b8-4ab2-93b2-3327a0bd38f0 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:21.075130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1179/2340] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:57:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:57:22.699713 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:57:31.066131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=38 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:57:31.610763 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:31.612304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:31.618537 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb669e-50b8-4ab2-93b2-3327a0bd38f0 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:31.625077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa9d08-0ab1-4dad-a63a-b7c7f051e07f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:31.630298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b66de-0bcb-4772-9253-978c82b887b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:31.630912 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b66de-0bcb-4772-9253-978c82b887b7 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:31.631188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b66de-0bcb-4772-9253-978c82b887b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:31.631457 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b66de-0bcb-4772-9253-978c82b887b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:31.633645 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5b4b66de-0bcb-4772-9253-978c82b887b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:31.638990 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb669e-50b8-4ab2-93b2-3327a0bd38f0 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:31.646934 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c12ff-ed7c-4fdb-b44b-6e64c7d2e14e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:31.648020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c12ff-ed7c-4fdb-b44b-6e64c7d2e14e service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:31.648577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c12ff-ed7c-4fdb-b44b-6e64c7d2e14e service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:31.649049 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c12ff-ed7c-4fdb-b44b-6e64c7d2e14e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:31.652126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-276c12ff-ed7c-4fdb-b44b-6e64c7d2e14e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:31.711454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b66de-0bcb-4772-9253-978c82b887b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:31.713572 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b66de-0bcb-4772-9253-978c82b887b7 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:31.723619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c12ff-ed7c-4fdb-b44b-6e64c7d2e14e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:31.725182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276c12ff-ed7c-4fdb-b44b-6e64c7d2e14e service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:31.725852 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1162/2341] 10.222.0.60 () {60 vars in 1396 bytes} [Mon Sep 19 14:57:31 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3268 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:57:31.735035 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1180/2342] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:57:31 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:57:40.424989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b66de-0bcb-4772-9253-978c82b887b7 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:40.426396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5b4b66de-0bcb-4772-9253-978c82b887b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:40.443269 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b66de-0bcb-4772-9253-978c82b887b7 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:40.449020 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45916209-2191-466f-ab6b-2d3962f03e3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:40.449650 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45916209-2191-466f-ab6b-2d3962f03e3a service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:40.449934 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45916209-2191-466f-ab6b-2d3962f03e3a service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:40.450212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45916209-2191-466f-ab6b-2d3962f03e3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:40.452046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-45916209-2191-466f-ab6b-2d3962f03e3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:40.524880 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45916209-2191-466f-ab6b-2d3962f03e3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:40.526486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45916209-2191-466f-ab6b-2d3962f03e3a service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:40.536224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1163/2343] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:57:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:57:40.538418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c12ff-ed7c-4fdb-b44b-6e64c7d2e14e service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:40.539852 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-276c12ff-ed7c-4fdb-b44b-6e64c7d2e14e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:40.607666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c12ff-ed7c-4fdb-b44b-6e64c7d2e14e service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:40.611000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c0c7f-80b3-4244-94ad-485c977304b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:40.611619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c0c7f-80b3-4244-94ad-485c977304b5 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:40.611952 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c0c7f-80b3-4244-94ad-485c977304b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:40.612365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c0c7f-80b3-4244-94ad-485c977304b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:40.622507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c0c7f-80b3-4244-94ad-485c977304b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:40.623998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c0c7f-80b3-4244-94ad-485c977304b5 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:40.634715 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1181/2344] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:57:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:57:50.531735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=56 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:57:50.630828 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=54 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:57:50.829012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45916209-2191-466f-ab6b-2d3962f03e3a service nova] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:50.839017 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45916209-2191-466f-ab6b-2d3962f03e3a service nova] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:50.842695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ee1b5-a431-4d58-bcf1-8639ad4ae30d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:50.843311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ee1b5-a431-4d58-bcf1-8639ad4ae30d service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:50.843589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ee1b5-a431-4d58-bcf1-8639ad4ae30d service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:50.843893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ee1b5-a431-4d58-bcf1-8639ad4ae30d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:50.845799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-bc3ee1b5-a431-4d58-bcf1-8639ad4ae30d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:50.915764 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ee1b5-a431-4d58-bcf1-8639ad4ae30d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:50.917255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ee1b5-a431-4d58-bcf1-8639ad4ae30d service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:50.927546 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1164/2345] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:57:51.049990 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86924dd1-964e-4273-9251-068dd749ac06 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:51.050803 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86924dd1-964e-4273-9251-068dd749ac06 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:51.051086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86924dd1-964e-4273-9251-068dd749ac06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:51.071797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-86924dd1-964e-4273-9251-068dd749ac06 None None] MFA Rules not processed for user `61743a63def7451d9b739f3b17afe8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:57:51.073471 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-86924dd1-964e-4273-9251-068dd749ac06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:51.081238 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1182/2346] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:57:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:57:51.603559 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ee1b5-a431-4d58-bcf1-8639ad4ae30d service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:51.612893 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ee1b5-a431-4d58-bcf1-8639ad4ae30d service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:51.616555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0865a8d-a72d-44f0-976d-3471f3e2c538 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:51.617152 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0865a8d-a72d-44f0-976d-3471f3e2c538 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:51.617428 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0865a8d-a72d-44f0-976d-3471f3e2c538 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:51.617696 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0865a8d-a72d-44f0-976d-3471f3e2c538 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:51.619430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c0865a8d-a72d-44f0-976d-3471f3e2c538 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:51.630205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0865a8d-a72d-44f0-976d-3471f3e2c538 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:51.631627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0865a8d-a72d-44f0-976d-3471f3e2c538 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:51.639947 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1165/2347] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:57:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:57:52.062297 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86924dd1-964e-4273-9251-068dd749ac06 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:52.094137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0865a8d-a72d-44f0-976d-3471f3e2c538 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:52.095628 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c0865a8d-a72d-44f0-976d-3471f3e2c538 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:52.096289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86924dd1-964e-4273-9251-068dd749ac06 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:52.102096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8695739-7eee-4d08-94d3-85a5e4ae6707 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:52.102824 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8695739-7eee-4d08-94d3-85a5e4ae6707 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:52.103115 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8695739-7eee-4d08-94d3-85a5e4ae6707 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:52.103390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8695739-7eee-4d08-94d3-85a5e4ae6707 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:52.105184 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e8695739-7eee-4d08-94d3-85a5e4ae6707 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:52.189170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8695739-7eee-4d08-94d3-85a5e4ae6707 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:52.192214 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8695739-7eee-4d08-94d3-85a5e4ae6707 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:52.210654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1183/2348] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:57:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:57:52.217780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0865a8d-a72d-44f0-976d-3471f3e2c538 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:52.225047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a90f35-8257-4dc0-a6e1-1f6743226379 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '568189358b154b66a729fae2f0411d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:52.225460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a90f35-8257-4dc0-a6e1-1f6743226379 service placement] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:52.225777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a90f35-8257-4dc0-a6e1-1f6743226379 service placement] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:52.226120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a90f35-8257-4dc0-a6e1-1f6743226379 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:52.228368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-97a90f35-8257-4dc0-a6e1-1f6743226379 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:52.245383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a90f35-8257-4dc0-a6e1-1f6743226379 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:52.245949 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a90f35-8257-4dc0-a6e1-1f6743226379 service placement] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:52.258321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1166/2349] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:57:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:57:54.984105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8695739-7eee-4d08-94d3-85a5e4ae6707 service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:55.018454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8695739-7eee-4d08-94d3-85a5e4ae6707 service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:55.027768 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a90f35-8257-4dc0-a6e1-1f6743226379 service placement] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:55.030558 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-97a90f35-8257-4dc0-a6e1-1f6743226379 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:55.043163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a54e5-e955-458d-9a3f-59a56338883e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:55.045945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a54e5-e955-458d-9a3f-59a56338883e service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:55.047165 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a54e5-e955-458d-9a3f-59a56338883e service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:55.049879 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a90f35-8257-4dc0-a6e1-1f6743226379 service placement] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:55.051231 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a54e5-e955-458d-9a3f-59a56338883e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:55.059944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051cbeb-4f76-44f9-9059-461e7c3368bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '568189358b154b66a729fae2f0411d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:55.060382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-750a54e5-e955-458d-9a3f-59a56338883e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:55.061489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051cbeb-4f76-44f9-9059-461e7c3368bc service placement] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:55.062130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051cbeb-4f76-44f9-9059-461e7c3368bc service placement] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:55.063234 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051cbeb-4f76-44f9-9059-461e7c3368bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:55.069577 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5051cbeb-4f76-44f9-9059-461e7c3368bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:55.090101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a54e5-e955-458d-9a3f-59a56338883e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:55.092506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a54e5-e955-458d-9a3f-59a56338883e service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:55.104271 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051cbeb-4f76-44f9-9059-461e7c3368bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:55.106542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051cbeb-4f76-44f9-9059-461e7c3368bc service placement] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:55.121174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1167/2350] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:57:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:57:55.131973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051cbeb-4f76-44f9-9059-461e7c3368bc service placement] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:55.136595 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5051cbeb-4f76-44f9-9059-461e7c3368bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:55.139552 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1184/2351] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:57:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:57:55.154563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051cbeb-4f76-44f9-9059-461e7c3368bc service placement] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:55.168185 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42916345-aed6-447f-af58-5bc4a46b17d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:55.169382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42916345-aed6-447f-af58-5bc4a46b17d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:55.169766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42916345-aed6-447f-af58-5bc4a46b17d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:55.170169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42916345-aed6-447f-af58-5bc4a46b17d7 admin admin] PATH_INFO: `/v3/users/1cccf354894c4311b7710d8a65682eeb` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:55.171639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42916345-aed6-447f-af58-5bc4a46b17d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cccf354894c4311b7710d8a65682eeb)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:55.174566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42916345-aed6-447f-af58-5bc4a46b17d7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:55.234707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-42916345-aed6-447f-af58-5bc4a46b17d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cccf354894c4311b7710d8a65682eeb'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:55.240665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:57:55.244364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-42916345-aed6-447f-af58-5bc4a46b17d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cccf354894c4311b7710d8a65682eeb'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:55.262097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-42916345-aed6-447f-af58-5bc4a46b17d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cccf354894c4311b7710d8a65682eeb'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:55.270712 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-42916345-aed6-447f-af58-5bc4a46b17d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cccf354894c4311b7710d8a65682eeb'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:55.271352 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-42916345-aed6-447f-af58-5bc4a46b17d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cccf354894c4311b7710d8a65682eeb'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:55.277699 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1168/2352] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:57:55 2022] DELETE /identity/v3/users/1cccf354894c4311b7710d8a65682eeb => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:57:55.465266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a54e5-e955-458d-9a3f-59a56338883e service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:55.477774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a54e5-e955-458d-9a3f-59a56338883e service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:55.540083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff97bc-619b-43ed-ac62-87f4b4728847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:55.540710 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff97bc-619b-43ed-ac62-87f4b4728847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:55.541000 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff97bc-619b-43ed-ac62-87f4b4728847 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:55.541276 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff97bc-619b-43ed-ac62-87f4b4728847 admin admin] PATH_INFO: `/v3/projects/73c223bd20fc43eca4c1cd0921f7906d` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:55.542422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff97bc-619b-43ed-ac62-87f4b4728847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73c223bd20fc43eca4c1cd0921f7906d)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:55.544364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff97bc-619b-43ed-ac62-87f4b4728847 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:55.594464 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5fff97bc-619b-43ed-ac62-87f4b4728847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73c223bd20fc43eca4c1cd0921f7906d', 'request_id': 'req-5fff97bc-619b-43ed-ac62-87f4b4728847'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:55.600908 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5fff97bc-619b-43ed-ac62-87f4b4728847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73c223bd20fc43eca4c1cd0921f7906d', 'request_id': 'req-5fff97bc-619b-43ed-ac62-87f4b4728847'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:55.607625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5fff97bc-619b-43ed-ac62-87f4b4728847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73c223bd20fc43eca4c1cd0921f7906d', 'request_id': 'req-5fff97bc-619b-43ed-ac62-87f4b4728847'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:55.622483 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5fff97bc-619b-43ed-ac62-87f4b4728847 admin admin] The token cache is being invalidate because project 73c223bd20fc43eca4c1cd0921f7906d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:57:55.622794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5fff97bc-619b-43ed-ac62-87f4b4728847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:55.623443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-5fff97bc-619b-43ed-ac62-87f4b4728847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:57:55.624993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1185/2353] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:57:55 2022] DELETE /identity/v3/projects/73c223bd20fc43eca4c1cd0921f7906d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:57:55.631094 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544a63a-e854-4255-a70b-40bc61936fc1 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:55.631457 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544a63a-e854-4255-a70b-40bc61936fc1 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:55.631840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544a63a-e854-4255-a70b-40bc61936fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:55.653671 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c544a63a-e854-4255-a70b-40bc61936fc1 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:57:55.654969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c544a63a-e854-4255-a70b-40bc61936fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:55.728053 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1169/2354] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:57:55.735671 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e5b2a-21c0-4725-9bc0-856ceac4e172 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:55.735985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e5b2a-21c0-4725-9bc0-856ceac4e172 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:55.736266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558e5b2a-21c0-4725-9bc0-856ceac4e172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:55.757755 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-558e5b2a-21c0-4725-9bc0-856ceac4e172 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:57:55.759118 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-558e5b2a-21c0-4725-9bc0-856ceac4e172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:55.769673 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1186/2355] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:57:55.774536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544a63a-e854-4255-a70b-40bc61936fc1 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:55.785496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544a63a-e854-4255-a70b-40bc61936fc1 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:55.790035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16324928-283e-4269-b903-2315aea60efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:55.790642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16324928-283e-4269-b903-2315aea60efb admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:55.790917 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16324928-283e-4269-b903-2315aea60efb admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:55.791188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16324928-283e-4269-b903-2315aea60efb admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:55.791847 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16324928-283e-4269-b903-2315aea60efb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:57:55.792153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16324928-283e-4269-b903-2315aea60efb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:55.793751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16324928-283e-4269-b903-2315aea60efb admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:55.806358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1170/2356] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:57:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:57:55.811016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e5b2a-21c0-4725-9bc0-856ceac4e172 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:55.815551 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:57:55.822005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558e5b2a-21c0-4725-9bc0-856ceac4e172 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:55.828649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c1522-a82a-438a-b35e-7cbf96678a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:55.829110 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c1522-a82a-438a-b35e-7cbf96678a47 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:55.829398 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c1522-a82a-438a-b35e-7cbf96678a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:55.829670 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c1522-a82a-438a-b35e-7cbf96678a47 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:55.830295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c1522-a82a-438a-b35e-7cbf96678a47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:55.832780 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c1522-a82a-438a-b35e-7cbf96678a47 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:55.861190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1187/2357] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:57:55 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:57:55.865966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16324928-283e-4269-b903-2315aea60efb admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:55.876967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16324928-283e-4269-b903-2315aea60efb admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:55.950606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb49a8-919c-4702-8b8e-7d9480d7651e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:55.951346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb49a8-919c-4702-8b8e-7d9480d7651e admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:55.951779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb49a8-919c-4702-8b8e-7d9480d7651e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:55.952183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb49a8-919c-4702-8b8e-7d9480d7651e admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:55.952936 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb49a8-919c-4702-8b8e-7d9480d7651e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:55.957952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb49a8-919c-4702-8b8e-7d9480d7651e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:55.987357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1171/2358] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:57:55 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:57:55.989225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c1522-a82a-438a-b35e-7cbf96678a47 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:55.999597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c1522-a82a-438a-b35e-7cbf96678a47 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:56.004543 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae24dc-b1cc-48ad-881b-1e596dfc532a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:56.005155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae24dc-b1cc-48ad-881b-1e596dfc532a admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:56.005433 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae24dc-b1cc-48ad-881b-1e596dfc532a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:56.005703 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae24dc-b1cc-48ad-881b-1e596dfc532a admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:56.006336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae24dc-b1cc-48ad-881b-1e596dfc532a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:56.008042 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae24dc-b1cc-48ad-881b-1e596dfc532a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:56.018333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1188/2359] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:57:56.023334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb49a8-919c-4702-8b8e-7d9480d7651e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:56.035486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb49a8-919c-4702-8b8e-7d9480d7651e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:56.040426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20fbf0-b544-4110-810b-f51d9d43937b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:56.041040 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20fbf0-b544-4110-810b-f51d9d43937b admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:56.041309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20fbf0-b544-4110-810b-f51d9d43937b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:56.041583 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20fbf0-b544-4110-810b-f51d9d43937b admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:56.042227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20fbf0-b544-4110-810b-f51d9d43937b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:56.043865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20fbf0-b544-4110-810b-f51d9d43937b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:56.055610 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1172/2360] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:57:56.060361 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae24dc-b1cc-48ad-881b-1e596dfc532a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:56.071162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae24dc-b1cc-48ad-881b-1e596dfc532a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:56.076138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6d796-add1-4930-94b9-f003a0c7205d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:56.076956 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6d796-add1-4930-94b9-f003a0c7205d admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:56.077271 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6d796-add1-4930-94b9-f003a0c7205d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:56.077617 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6d796-add1-4930-94b9-f003a0c7205d admin admin] PATH_INFO: `/v3/projects/db382ca794b542e7a00b15ce60ae12e0/users/d5a2824f8161498fad8c6312a5112f5b/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:56.096752 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6d796-add1-4930-94b9-f003a0c7205d admin admin] RBAC: Authorizing `identity:create_grant(project_id=db382ca794b542e7a00b15ce60ae12e0, user_id=d5a2824f8161498fad8c6312a5112f5b, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:56.100295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6d796-add1-4930-94b9-f003a0c7205d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:56.117903 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1189/2361] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:57:56 2022] PUT /identity/v3/projects/db382ca794b542e7a00b15ce60ae12e0/users/d5a2824f8161498fad8c6312a5112f5b/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:57:56.129689 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad59e9-5757-4f78-92c4-d55924a9c9b4 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:56.130057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad59e9-5757-4f78-92c4-d55924a9c9b4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:56.130345 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad59e9-5757-4f78-92c4-d55924a9c9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:56.160040 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-84ad59e9-5757-4f78-92c4-d55924a9c9b4 None None] MFA Rules not processed for user `d5a2824f8161498fad8c6312a5112f5b`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:57:56.161536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-84ad59e9-5757-4f78-92c4-d55924a9c9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:56.188896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:57:56.235017 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1173/2362] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:57:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:57:56.311191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6d796-add1-4930-94b9-f003a0c7205d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:57:56.313113 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d7c6d796-add1-4930-94b9-f003a0c7205d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:57:56.388972 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6d796-add1-4930-94b9-f003a0c7205d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:57:56.395215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1f5cb8-8a19-41ea-b66e-09a02d74cc25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:57:56.395744 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f5cb8-8a19-41ea-b66e-09a02d74cc25 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:57:56.396058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f5cb8-8a19-41ea-b66e-09a02d74cc25 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:57:56.396334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1f5cb8-8a19-41ea-b66e-09a02d74cc25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:57:56.404785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f5cb8-8a19-41ea-b66e-09a02d74cc25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:57:56.406278 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1f5cb8-8a19-41ea-b66e-09a02d74cc25 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:57:56.416249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1190/2363] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:57:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:04.059409 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad59e9-5757-4f78-92c4-d55924a9c9b4 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:04.124731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad59e9-5757-4f78-92c4-d55924a9c9b4 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:04.129854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd7cf8-2026-414a-a608-42c53d294d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:04.130483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd7cf8-2026-414a-a608-42c53d294d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:04.130765 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd7cf8-2026-414a-a608-42c53d294d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:04.131040 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dd7cf8-2026-414a-a608-42c53d294d35 admin admin] PATH_INFO: `/v3/users/d5a2824f8161498fad8c6312a5112f5b` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:04.132185 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd7cf8-2026-414a-a608-42c53d294d35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5a2824f8161498fad8c6312a5112f5b)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:04.134184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dd7cf8-2026-414a-a608-42c53d294d35 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:04.183249 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-26dd7cf8-2026-414a-a608-42c53d294d35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5a2824f8161498fad8c6312a5112f5b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:04.191100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-26dd7cf8-2026-414a-a608-42c53d294d35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5a2824f8161498fad8c6312a5112f5b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:04.203044 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-26dd7cf8-2026-414a-a608-42c53d294d35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5a2824f8161498fad8c6312a5112f5b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:04.209702 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-26dd7cf8-2026-414a-a608-42c53d294d35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5a2824f8161498fad8c6312a5112f5b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:04.210537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-26dd7cf8-2026-414a-a608-42c53d294d35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5a2824f8161498fad8c6312a5112f5b'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:04.216714 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1174/2364] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:58:04 2022] DELETE /identity/v3/users/d5a2824f8161498fad8c6312a5112f5b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:04.221214 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de239def-429f-45d5-a6ad-495a06367d35 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:04.222306 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de239def-429f-45d5-a6ad-495a06367d35 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:04.222306 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de239def-429f-45d5-a6ad-495a06367d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:04.242429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-de239def-429f-45d5-a6ad-495a06367d35 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:04.243635 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-de239def-429f-45d5-a6ad-495a06367d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:04.303247 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1191/2365] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:04.313441 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd7cf8-2026-414a-a608-42c53d294d35 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:04.315040 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-26dd7cf8-2026-414a-a608-42c53d294d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:04.361643 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:04.374587 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dd7cf8-2026-414a-a608-42c53d294d35 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:04.378345 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307ec16-5200-47cd-964d-5ac470b8e07c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:04.378979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307ec16-5200-47cd-964d-5ac470b8e07c service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:04.379261 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307ec16-5200-47cd-964d-5ac470b8e07c service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:04.379533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307ec16-5200-47cd-964d-5ac470b8e07c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:04.388595 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307ec16-5200-47cd-964d-5ac470b8e07c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:04.390038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307ec16-5200-47cd-964d-5ac470b8e07c service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:04.400107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1175/2366] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:04.743683 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de239def-429f-45d5-a6ad-495a06367d35 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:04.745183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-de239def-429f-45d5-a6ad-495a06367d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:04.757435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de239def-429f-45d5-a6ad-495a06367d35 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:04.762008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f05c3d-5882-41c2-846c-886fd19e6890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:04.762635 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f05c3d-5882-41c2-846c-886fd19e6890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:04.762914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f05c3d-5882-41c2-846c-886fd19e6890 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:04.763192 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f05c3d-5882-41c2-846c-886fd19e6890 admin admin] PATH_INFO: `/v3/projects/db382ca794b542e7a00b15ce60ae12e0` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:04.764319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f05c3d-5882-41c2-846c-886fd19e6890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db382ca794b542e7a00b15ce60ae12e0)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:04.766238 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f05c3d-5882-41c2-846c-886fd19e6890 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:04.804559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:04.822991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-82f05c3d-5882-41c2-846c-886fd19e6890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db382ca794b542e7a00b15ce60ae12e0', 'request_id': 'req-82f05c3d-5882-41c2-846c-886fd19e6890'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:04.828520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-82f05c3d-5882-41c2-846c-886fd19e6890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db382ca794b542e7a00b15ce60ae12e0', 'request_id': 'req-82f05c3d-5882-41c2-846c-886fd19e6890'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:04.835524 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-82f05c3d-5882-41c2-846c-886fd19e6890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db382ca794b542e7a00b15ce60ae12e0', 'request_id': 'req-82f05c3d-5882-41c2-846c-886fd19e6890'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:04.848075 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-82f05c3d-5882-41c2-846c-886fd19e6890 admin admin] The token cache is being invalidate because project db382ca794b542e7a00b15ce60ae12e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:58:04.848365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-82f05c3d-5882-41c2-846c-886fd19e6890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:04.848992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-82f05c3d-5882-41c2-846c-886fd19e6890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:04.850502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1192/2367] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:58:04 2022] DELETE /identity/v3/projects/db382ca794b542e7a00b15ce60ae12e0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:04.855356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a074707-917c-4a09-a269-881a6ef4b8ba None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:04.855647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a074707-917c-4a09-a269-881a6ef4b8ba None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:04.855973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a074707-917c-4a09-a269-881a6ef4b8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:04.875508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-7a074707-917c-4a09-a269-881a6ef4b8ba None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:04.876886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7a074707-917c-4a09-a269-881a6ef4b8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:04.936977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1176/2368] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:04.945542 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3199a7-40b6-44fb-b68f-94c2037c75fc None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:04.945788 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3199a7-40b6-44fb-b68f-94c2037c75fc None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:04.946069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3199a7-40b6-44fb-b68f-94c2037c75fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:04.967790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-7a3199a7-40b6-44fb-b68f-94c2037c75fc None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:04.969088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7a3199a7-40b6-44fb-b68f-94c2037c75fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:04.977264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1193/2369] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:04.981612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a074707-917c-4a09-a269-881a6ef4b8ba None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:04.991480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a074707-917c-4a09-a269-881a6ef4b8ba None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:04.996194 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7bb1b-9ffe-418b-810b-1527d453a4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:04.996803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7bb1b-9ffe-418b-810b-1527d453a4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:04.997085 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7bb1b-9ffe-418b-810b-1527d453a4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:04.997359 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7bb1b-9ffe-418b-810b-1527d453a4e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:04.998026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7bb1b-9ffe-418b-810b-1527d453a4e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:58:04.998327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7bb1b-9ffe-418b-810b-1527d453a4e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:04.999971 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7bb1b-9ffe-418b-810b-1527d453a4e4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:05.015390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1177/2370] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:58:05.020009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3199a7-40b6-44fb-b68f-94c2037c75fc None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:05.029353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3199a7-40b6-44fb-b68f-94c2037c75fc None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:05.034337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf9387-18cf-4ec6-b9c0-f1d4c918f928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:05.034944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf9387-18cf-4ec6-b9c0-f1d4c918f928 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:05.035228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf9387-18cf-4ec6-b9c0-f1d4c918f928 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:05.035488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf9387-18cf-4ec6-b9c0-f1d4c918f928 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:05.037096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf9387-18cf-4ec6-b9c0-f1d4c918f928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:05.039132 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf9387-18cf-4ec6-b9c0-f1d4c918f928 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:05.078307 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1194/2371] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:58:05 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:05.083204 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7bb1b-9ffe-418b-810b-1527d453a4e4 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:05.096133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7bb1b-9ffe-418b-810b-1527d453a4e4 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:05.159375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd36b42-7942-45a9-ae0f-6729fe509662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:05.160013 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd36b42-7942-45a9-ae0f-6729fe509662 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:05.160302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd36b42-7942-45a9-ae0f-6729fe509662 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:05.160575 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd36b42-7942-45a9-ae0f-6729fe509662 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:05.161198 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd36b42-7942-45a9-ae0f-6729fe509662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:05.163275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd36b42-7942-45a9-ae0f-6729fe509662 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:05.187450 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1178/2372] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:58:05 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:05.192196 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf9387-18cf-4ec6-b9c0-f1d4c918f928 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:05.202457 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf9387-18cf-4ec6-b9c0-f1d4c918f928 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:05.207709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a045187-bc58-4045-b7ca-f5cb0ab6474a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:05.208342 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a045187-bc58-4045-b7ca-f5cb0ab6474a admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:05.208624 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a045187-bc58-4045-b7ca-f5cb0ab6474a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:05.208897 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a045187-bc58-4045-b7ca-f5cb0ab6474a admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:05.209532 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a045187-bc58-4045-b7ca-f5cb0ab6474a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:05.211128 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a045187-bc58-4045-b7ca-f5cb0ab6474a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:05.220931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1195/2373] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:58:05.224915 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd36b42-7942-45a9-ae0f-6729fe509662 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:05.234735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd36b42-7942-45a9-ae0f-6729fe509662 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:05.239416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b133912-669a-4b32-a174-1abc1a9f293e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:05.240034 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b133912-669a-4b32-a174-1abc1a9f293e admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:05.240314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b133912-669a-4b32-a174-1abc1a9f293e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:05.240583 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b133912-669a-4b32-a174-1abc1a9f293e admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:05.241210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b133912-669a-4b32-a174-1abc1a9f293e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:05.242788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b133912-669a-4b32-a174-1abc1a9f293e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:05.254060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1179/2374] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:05.258314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a045187-bc58-4045-b7ca-f5cb0ab6474a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:05.269087 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a045187-bc58-4045-b7ca-f5cb0ab6474a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:05.273834 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5725b7b-6d17-488d-815c-a9cc31887dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:05.274466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5725b7b-6d17-488d-815c-a9cc31887dde admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:05.274751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5725b7b-6d17-488d-815c-a9cc31887dde admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:05.275056 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5725b7b-6d17-488d-815c-a9cc31887dde admin admin] PATH_INFO: `/v3/projects/b67420ef57384c4f8e9a89987201e314/users/03a6600939a844e2a736726f2ddca24d/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:05.290056 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5725b7b-6d17-488d-815c-a9cc31887dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=b67420ef57384c4f8e9a89987201e314, user_id=03a6600939a844e2a736726f2ddca24d, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:05.292824 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5725b7b-6d17-488d-815c-a9cc31887dde admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:05.306740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1196/2375] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:58:05 2022] PUT /identity/v3/projects/b67420ef57384c4f8e9a89987201e314/users/03a6600939a844e2a736726f2ddca24d/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:05.317668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870c26a-a258-4d29-b228-35e5206da12c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:05.317955 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870c26a-a258-4d29-b228-35e5206da12c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:05.318228 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870c26a-a258-4d29-b228-35e5206da12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:05.345985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8870c26a-a258-4d29-b228-35e5206da12c None None] MFA Rules not processed for user `03a6600939a844e2a736726f2ddca24d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:05.347723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8870c26a-a258-4d29-b228-35e5206da12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:05.389309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:05.413739 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1180/2376] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:05.418391 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5725b7b-6d17-488d-815c-a9cc31887dde admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:05.481903 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5725b7b-6d17-488d-815c-a9cc31887dde admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:05.486908 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cbeccd-680b-4d6a-aca1-6623979eced9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:05.487619 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbeccd-680b-4d6a-aca1-6623979eced9 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:05.487927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbeccd-680b-4d6a-aca1-6623979eced9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:05.488207 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbeccd-680b-4d6a-aca1-6623979eced9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:05.488836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cbeccd-680b-4d6a-aca1-6623979eced9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:05.490735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cbeccd-680b-4d6a-aca1-6623979eced9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:05.517390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1197/2377] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:58:05 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:58:05.524807 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870c26a-a258-4d29-b228-35e5206da12c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:05.536668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870c26a-a258-4d29-b228-35e5206da12c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:05.593700 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab19d4-40ef-43a1-b0b5-2e63540365e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:05.594306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab19d4-40ef-43a1-b0b5-2e63540365e9 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:05.594587 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab19d4-40ef-43a1-b0b5-2e63540365e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:05.594861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab19d4-40ef-43a1-b0b5-2e63540365e9 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:05.595480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab19d4-40ef-43a1-b0b5-2e63540365e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:05.598260 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab19d4-40ef-43a1-b0b5-2e63540365e9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:05.619806 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1181/2378] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:58:05 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:05.623812 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cbeccd-680b-4d6a-aca1-6623979eced9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:05.634089 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cbeccd-680b-4d6a-aca1-6623979eced9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:05.638621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4be84-8aac-4312-8dda-b14ea4e87ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:05.639217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4be84-8aac-4312-8dda-b14ea4e87ced admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:05.639495 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4be84-8aac-4312-8dda-b14ea4e87ced admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:05.639762 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4be84-8aac-4312-8dda-b14ea4e87ced admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:05.640422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4be84-8aac-4312-8dda-b14ea4e87ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:05.642022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4be84-8aac-4312-8dda-b14ea4e87ced admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:05.653849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1198/2379] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:05.658131 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab19d4-40ef-43a1-b0b5-2e63540365e9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:05.668718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab19d4-40ef-43a1-b0b5-2e63540365e9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:05.673956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0736b6-2815-400e-97d3-0b9e3e654b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:05.674655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0736b6-2815-400e-97d3-0b9e3e654b74 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:05.674972 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0736b6-2815-400e-97d3-0b9e3e654b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:05.675271 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0736b6-2815-400e-97d3-0b9e3e654b74 admin admin] PATH_INFO: `/v3/projects/b2955dad099146e7a7a6802c0237986b/users/02ac33167595477ea24fe7c7bb2a08a2/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:05.691245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0736b6-2815-400e-97d3-0b9e3e654b74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2955dad099146e7a7a6802c0237986b, user_id=02ac33167595477ea24fe7c7bb2a08a2, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:05.693950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0736b6-2815-400e-97d3-0b9e3e654b74 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:05.707406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1182/2380] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:58:05 2022] PUT /identity/v3/projects/b2955dad099146e7a7a6802c0237986b/users/02ac33167595477ea24fe7c7bb2a08a2/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:05.711480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4be84-8aac-4312-8dda-b14ea4e87ced admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:05.721800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4be84-8aac-4312-8dda-b14ea4e87ced admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:05.743380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:05.782746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcfce7-1f2e-4374-b5a1-0c1c00889913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:05.783359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcfce7-1f2e-4374-b5a1-0c1c00889913 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:05.783649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcfce7-1f2e-4374-b5a1-0c1c00889913 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:05.783956 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcfce7-1f2e-4374-b5a1-0c1c00889913 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:05.784592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadcfce7-1f2e-4374-b5a1-0c1c00889913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:05.786216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadcfce7-1f2e-4374-b5a1-0c1c00889913 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:05.796709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1199/2381] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:05.800845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0736b6-2815-400e-97d3-0b9e3e654b74 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:05.811779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0736b6-2815-400e-97d3-0b9e3e654b74 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:05.816994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01e5e6-53fa-4fba-beee-ffda3543472a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:05.817648 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01e5e6-53fa-4fba-beee-ffda3543472a admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:05.817957 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01e5e6-53fa-4fba-beee-ffda3543472a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:05.818232 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01e5e6-53fa-4fba-beee-ffda3543472a admin admin] PATH_INFO: `/v3/projects/b2955dad099146e7a7a6802c0237986b/users/02ac33167595477ea24fe7c7bb2a08a2/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:05.820185 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01e5e6-53fa-4fba-beee-ffda3543472a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2955dad099146e7a7a6802c0237986b, user_id=02ac33167595477ea24fe7c7bb2a08a2, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:05.822835 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed01e5e6-53fa-4fba-beee-ffda3543472a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:05.834686 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1183/2382] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:58:05 2022] PUT /identity/v3/projects/b2955dad099146e7a7a6802c0237986b/users/02ac33167595477ea24fe7c7bb2a08a2/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:05.846775 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee79af-8e89-4f52-b9fc-9c1fe8d7b9dd None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:05.847079 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee79af-8e89-4f52-b9fc-9c1fe8d7b9dd None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:05.847387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee79af-8e89-4f52-b9fc-9c1fe8d7b9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:05.877180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f7ee79af-8e89-4f52-b9fc-9c1fe8d7b9dd None None] MFA Rules not processed for user `02ac33167595477ea24fe7c7bb2a08a2`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:05.878636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f7ee79af-8e89-4f52-b9fc-9c1fe8d7b9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:05.955170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1200/2383] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:05.967096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01e5e6-53fa-4fba-beee-ffda3543472a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:05.968890 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ed01e5e6-53fa-4fba-beee-ffda3543472a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:06.031356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01e5e6-53fa-4fba-beee-ffda3543472a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:06.035495 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23ca33-ef21-4bf7-9152-8462a4dd55ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:06.036141 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23ca33-ef21-4bf7-9152-8462a4dd55ab service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:06.036422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23ca33-ef21-4bf7-9152-8462a4dd55ab service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:06.036692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23ca33-ef21-4bf7-9152-8462a4dd55ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:06.046320 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23ca33-ef21-4bf7-9152-8462a4dd55ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:06.047750 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23ca33-ef21-4bf7-9152-8462a4dd55ab service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:06.058034 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1184/2384] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:05 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:06.461333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee79af-8e89-4f52-b9fc-9c1fe8d7b9dd None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:06.528070 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee79af-8e89-4f52-b9fc-9c1fe8d7b9dd None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:06.535294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9f9ec-2464-4d34-b7c1-57058a293549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:06.535995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9f9ec-2464-4d34-b7c1-57058a293549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:06.536277 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9f9ec-2464-4d34-b7c1-57058a293549 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:06.536555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9f9ec-2464-4d34-b7c1-57058a293549 admin admin] PATH_INFO: `/v3/users/03a6600939a844e2a736726f2ddca24d` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:06.537710 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9f9ec-2464-4d34-b7c1-57058a293549 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03a6600939a844e2a736726f2ddca24d)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:06.539644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a9f9ec-2464-4d34-b7c1-57058a293549 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:06.590426 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-38a9f9ec-2464-4d34-b7c1-57058a293549 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03a6600939a844e2a736726f2ddca24d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:06.591945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:06.601446 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-38a9f9ec-2464-4d34-b7c1-57058a293549 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03a6600939a844e2a736726f2ddca24d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:06.614472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-38a9f9ec-2464-4d34-b7c1-57058a293549 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03a6600939a844e2a736726f2ddca24d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:06.622617 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-38a9f9ec-2464-4d34-b7c1-57058a293549 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03a6600939a844e2a736726f2ddca24d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:06.623284 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-38a9f9ec-2464-4d34-b7c1-57058a293549 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03a6600939a844e2a736726f2ddca24d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:06.629603 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1201/2385] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:58:06 2022] DELETE /identity/v3/users/03a6600939a844e2a736726f2ddca24d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:06.634161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426aa3b-fc5f-4f3e-a3f9-ca7f635f58a8 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:06.634637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426aa3b-fc5f-4f3e-a3f9-ca7f635f58a8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:06.634735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426aa3b-fc5f-4f3e-a3f9-ca7f635f58a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:06.656348 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2426aa3b-fc5f-4f3e-a3f9-ca7f635f58a8 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:06.657580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2426aa3b-fc5f-4f3e-a3f9-ca7f635f58a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:06.695688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:06.726923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1185/2386] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:06.737915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9f9ec-2464-4d34-b7c1-57058a293549 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:06.740760 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-38a9f9ec-2464-4d34-b7c1-57058a293549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:06.799068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9f9ec-2464-4d34-b7c1-57058a293549 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:06.803502 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c55fec-66e6-4739-8147-6fe735cce67e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:06.804154 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c55fec-66e6-4739-8147-6fe735cce67e service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:06.804435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c55fec-66e6-4739-8147-6fe735cce67e service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:06.804706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c55fec-66e6-4739-8147-6fe735cce67e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:06.814637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c55fec-66e6-4739-8147-6fe735cce67e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:06.817400 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c55fec-66e6-4739-8147-6fe735cce67e service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:06.827878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1202/2387] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:58:07.589848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426aa3b-fc5f-4f3e-a3f9-ca7f635f58a8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:07.591466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2426aa3b-fc5f-4f3e-a3f9-ca7f635f58a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:07.607708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426aa3b-fc5f-4f3e-a3f9-ca7f635f58a8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:07.613566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87814a5b-f666-4d41-b010-e900056f69b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:07.614211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87814a5b-f666-4d41-b010-e900056f69b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:07.614489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87814a5b-f666-4d41-b010-e900056f69b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:07.614762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87814a5b-f666-4d41-b010-e900056f69b6 admin admin] PATH_INFO: `/v3/projects/b67420ef57384c4f8e9a89987201e314` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:07.616017 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87814a5b-f666-4d41-b010-e900056f69b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b67420ef57384c4f8e9a89987201e314)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:07.617919 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87814a5b-f666-4d41-b010-e900056f69b6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:07.668635 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-87814a5b-f666-4d41-b010-e900056f69b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b67420ef57384c4f8e9a89987201e314', 'request_id': 'req-87814a5b-f666-4d41-b010-e900056f69b6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:07.675690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-87814a5b-f666-4d41-b010-e900056f69b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b67420ef57384c4f8e9a89987201e314', 'request_id': 'req-87814a5b-f666-4d41-b010-e900056f69b6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:07.684631 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-87814a5b-f666-4d41-b010-e900056f69b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b67420ef57384c4f8e9a89987201e314', 'request_id': 'req-87814a5b-f666-4d41-b010-e900056f69b6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:07.696540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-87814a5b-f666-4d41-b010-e900056f69b6 admin admin] The token cache is being invalidate because project b67420ef57384c4f8e9a89987201e314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:58:07.696811 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-87814a5b-f666-4d41-b010-e900056f69b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:07.697460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-87814a5b-f666-4d41-b010-e900056f69b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:07.698950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1186/2388] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:58:07 2022] DELETE /identity/v3/projects/b67420ef57384c4f8e9a89987201e314 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:07.703091 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c55fec-66e6-4739-8147-6fe735cce67e service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:07.713490 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c55fec-66e6-4739-8147-6fe735cce67e service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:07.774729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463f7fa-feec-407a-a754-485b58dbbf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:07.774729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463f7fa-feec-407a-a754-485b58dbbf9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:07.774984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463f7fa-feec-407a-a754-485b58dbbf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:07.775969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463f7fa-feec-407a-a754-485b58dbbf9d admin admin] PATH_INFO: `/v3/users/02ac33167595477ea24fe7c7bb2a08a2` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:07.776820 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463f7fa-feec-407a-a754-485b58dbbf9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=02ac33167595477ea24fe7c7bb2a08a2)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:07.778756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e463f7fa-feec-407a-a754-485b58dbbf9d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:07.819061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87814a5b-f666-4d41-b010-e900056f69b6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:07.821336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-87814a5b-f666-4d41-b010-e900056f69b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:07.844536 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e463f7fa-feec-407a-a754-485b58dbbf9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02ac33167595477ea24fe7c7bb2a08a2'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:07.854355 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e463f7fa-feec-407a-a754-485b58dbbf9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02ac33167595477ea24fe7c7bb2a08a2'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:07.869609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:07.873391 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e463f7fa-feec-407a-a754-485b58dbbf9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02ac33167595477ea24fe7c7bb2a08a2'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:07.887246 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e463f7fa-feec-407a-a754-485b58dbbf9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02ac33167595477ea24fe7c7bb2a08a2'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:07.888301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-e463f7fa-feec-407a-a754-485b58dbbf9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02ac33167595477ea24fe7c7bb2a08a2'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:07.898207 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1203/2389] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:58:07 2022] DELETE /identity/v3/users/02ac33167595477ea24fe7c7bb2a08a2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:07.922102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87814a5b-f666-4d41-b010-e900056f69b6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:07.986563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:08.001508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7bf91-a967-4809-9b68-9a93b1c4a550 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:08.002298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7bf91-a967-4809-9b68-9a93b1c4a550 service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:08.002637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7bf91-a967-4809-9b68-9a93b1c4a550 service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:08.002926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7bf91-a967-4809-9b68-9a93b1c4a550 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:08.006560 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-88b7bf91-a967-4809-9b68-9a93b1c4a550 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:08.018246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b7bf91-a967-4809-9b68-9a93b1c4a550 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:08.020134 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b7bf91-a967-4809-9b68-9a93b1c4a550 service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:08.032481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1187/2390] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:58:08.370613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463f7fa-feec-407a-a754-485b58dbbf9d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:08.372188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e463f7fa-feec-407a-a754-485b58dbbf9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:08.453960 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463f7fa-feec-407a-a754-485b58dbbf9d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:08.459124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef28f6-50cd-423e-b2f5-560941271ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:08.459761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef28f6-50cd-423e-b2f5-560941271ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:08.460175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef28f6-50cd-423e-b2f5-560941271ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:08.460448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef28f6-50cd-423e-b2f5-560941271ea1 admin admin] PATH_INFO: `/v3/projects/b2955dad099146e7a7a6802c0237986b` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:08.461672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef28f6-50cd-423e-b2f5-560941271ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2955dad099146e7a7a6802c0237986b)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:08.463715 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ef28f6-50cd-423e-b2f5-560941271ea1 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:08.516652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f7ef28f6-50cd-423e-b2f5-560941271ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2955dad099146e7a7a6802c0237986b', 'request_id': 'req-f7ef28f6-50cd-423e-b2f5-560941271ea1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:08.523191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f7ef28f6-50cd-423e-b2f5-560941271ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2955dad099146e7a7a6802c0237986b', 'request_id': 'req-f7ef28f6-50cd-423e-b2f5-560941271ea1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:08.532847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f7ef28f6-50cd-423e-b2f5-560941271ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2955dad099146e7a7a6802c0237986b', 'request_id': 'req-f7ef28f6-50cd-423e-b2f5-560941271ea1'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:08.557262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f7ef28f6-50cd-423e-b2f5-560941271ea1 admin admin] The token cache is being invalidate because project b2955dad099146e7a7a6802c0237986b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:58:08.557851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f7ef28f6-50cd-423e-b2f5-560941271ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:08.558878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f7ef28f6-50cd-423e-b2f5-560941271ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:08.567493 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1204/2391] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:58:08 2022] DELETE /identity/v3/projects/b2955dad099146e7a7a6802c0237986b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:08.574108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12b141-6ab8-4708-b129-703c7f957488 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:08.574733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12b141-6ab8-4708-b129-703c7f957488 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:08.575794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12b141-6ab8-4708-b129-703c7f957488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:08.613000 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-7a12b141-6ab8-4708-b129-703c7f957488 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:08.614399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7a12b141-6ab8-4708-b129-703c7f957488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:08.702700 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1188/2392] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:08.711366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daad46-5fa9-4567-96c9-01955d702d5b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:08.711897 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daad46-5fa9-4567-96c9-01955d702d5b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:08.712001 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daad46-5fa9-4567-96c9-01955d702d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:08.742042 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c4daad46-5fa9-4567-96c9-01955d702d5b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:08.743362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c4daad46-5fa9-4567-96c9-01955d702d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:08.755614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1205/2393] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:08.756872 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12b141-6ab8-4708-b129-703c7f957488 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:08.769164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12b141-6ab8-4708-b129-703c7f957488 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:08.774317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cfbe9-925c-4e21-8f20-ce7bece1761d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:08.775438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cfbe9-925c-4e21-8f20-ce7bece1761d admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:08.775438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cfbe9-925c-4e21-8f20-ce7bece1761d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:08.775910 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252cfbe9-925c-4e21-8f20-ce7bece1761d admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:08.776672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cfbe9-925c-4e21-8f20-ce7bece1761d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:58:08.777074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cfbe9-925c-4e21-8f20-ce7bece1761d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:08.778782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252cfbe9-925c-4e21-8f20-ce7bece1761d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:08.805603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1189/2394] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:58:08.806435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4daad46-5fa9-4567-96c9-01955d702d5b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:08.815679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4daad46-5fa9-4567-96c9-01955d702d5b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:08.821064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bd384-8b5d-44f7-9ab3-a26c5c4b2674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:08.821933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bd384-8b5d-44f7-9ab3-a26c5c4b2674 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:08.822316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bd384-8b5d-44f7-9ab3-a26c5c4b2674 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:08.822706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bd384-8b5d-44f7-9ab3-a26c5c4b2674 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:08.823518 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bd384-8b5d-44f7-9ab3-a26c5c4b2674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:08.825675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bd384-8b5d-44f7-9ab3-a26c5c4b2674 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:08.860228 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1206/2395] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:58:08 2022] POST /identity/v3/projects => generated 401 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:08.864232 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cfbe9-925c-4e21-8f20-ce7bece1761d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:08.874860 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252cfbe9-925c-4e21-8f20-ce7bece1761d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:08.948497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550b00b-6584-4d47-8fa8-9ac48007e8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:08.949184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550b00b-6584-4d47-8fa8-9ac48007e8af admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:08.949467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550b00b-6584-4d47-8fa8-9ac48007e8af admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:08.949796 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550b00b-6584-4d47-8fa8-9ac48007e8af admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:08.950953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550b00b-6584-4d47-8fa8-9ac48007e8af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:08.952513 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550b00b-6584-4d47-8fa8-9ac48007e8af admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:08.978246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1190/2396] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:58:08 2022] POST /identity/v3/users => generated 301 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:08.979541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bd384-8b5d-44f7-9ab3-a26c5c4b2674 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:08.991466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bd384-8b5d-44f7-9ab3-a26c5c4b2674 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:08.996737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bdf1d-ce56-4fe0-b014-3e1ab373a627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:08.997377 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bdf1d-ce56-4fe0-b014-3e1ab373a627 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:08.997655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bdf1d-ce56-4fe0-b014-3e1ab373a627 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:08.997924 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bdf1d-ce56-4fe0-b014-3e1ab373a627 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:08.998558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01bdf1d-ce56-4fe0-b014-3e1ab373a627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:09.000235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01bdf1d-ce56-4fe0-b014-3e1ab373a627 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:09.010716 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1207/2397] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:09.014895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550b00b-6584-4d47-8fa8-9ac48007e8af admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:09.026438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550b00b-6584-4d47-8fa8-9ac48007e8af admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:09.031389 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae30075-703e-401d-8333-226988fac76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:09.032112 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae30075-703e-401d-8333-226988fac76b admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:09.032397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae30075-703e-401d-8333-226988fac76b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:09.032674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae30075-703e-401d-8333-226988fac76b admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:09.033423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae30075-703e-401d-8333-226988fac76b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:09.035422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae30075-703e-401d-8333-226988fac76b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:09.046660 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1191/2398] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:09.050936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bdf1d-ce56-4fe0-b014-3e1ab373a627 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:09.061669 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bdf1d-ce56-4fe0-b014-3e1ab373a627 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:09.066463 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a675ec-4737-4a70-a711-6829e4ff1b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:09.067112 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a675ec-4737-4a70-a711-6829e4ff1b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:09.067390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a675ec-4737-4a70-a711-6829e4ff1b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:09.067660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a675ec-4737-4a70-a711-6829e4ff1b9d admin admin] PATH_INFO: `/v3/projects/70c8cc1e4d9a4794ac587d52023bc1a3/users/a3afb901f13c40fca50007ca536b49b2/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:09.083004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a675ec-4737-4a70-a711-6829e4ff1b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c8cc1e4d9a4794ac587d52023bc1a3, user_id=a3afb901f13c40fca50007ca536b49b2, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:09.085708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a675ec-4737-4a70-a711-6829e4ff1b9d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:09.099160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1208/2399] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:58:09 2022] PUT /identity/v3/projects/70c8cc1e4d9a4794ac587d52023bc1a3/users/a3afb901f13c40fca50007ca536b49b2/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:09.110077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b660da9-de02-48e4-9d8d-84f01ddb3932 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:09.110077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b660da9-de02-48e4-9d8d-84f01ddb3932 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:09.110077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b660da9-de02-48e4-9d8d-84f01ddb3932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:09.137530 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5b660da9-de02-48e4-9d8d-84f01ddb3932 None None] MFA Rules not processed for user `a3afb901f13c40fca50007ca536b49b2`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:09.138774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5b660da9-de02-48e4-9d8d-84f01ddb3932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:09.179045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:09.204891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1192/2400] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:09 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:09.210292 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a675ec-4737-4a70-a711-6829e4ff1b9d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:09.224649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:09.275360 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a675ec-4737-4a70-a711-6829e4ff1b9d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:09.280258 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f48d7-69ef-49de-a8fd-55692cf2b8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:09.280802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f48d7-69ef-49de-a8fd-55692cf2b8bb admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:09.281085 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f48d7-69ef-49de-a8fd-55692cf2b8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:09.281360 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f48d7-69ef-49de-a8fd-55692cf2b8bb admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:09.281991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f48d7-69ef-49de-a8fd-55692cf2b8bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:09.284521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f48d7-69ef-49de-a8fd-55692cf2b8bb admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:09.310917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1209/2401] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:58:09 2022] POST /identity/v3/projects => generated 395 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:09.315229 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b660da9-de02-48e4-9d8d-84f01ddb3932 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:09.326254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b660da9-de02-48e4-9d8d-84f01ddb3932 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:09.386025 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592598d-732b-4ee2-aa9b-c7fba35c2a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:09.386644 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592598d-732b-4ee2-aa9b-c7fba35c2a40 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:09.386924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592598d-732b-4ee2-aa9b-c7fba35c2a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:09.387194 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4592598d-732b-4ee2-aa9b-c7fba35c2a40 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:09.387928 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592598d-732b-4ee2-aa9b-c7fba35c2a40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:09.389780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4592598d-732b-4ee2-aa9b-c7fba35c2a40 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:09.411344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1193/2402] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:58:09 2022] POST /identity/v3/users => generated 304 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:09.415371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f48d7-69ef-49de-a8fd-55692cf2b8bb admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:09.426414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f48d7-69ef-49de-a8fd-55692cf2b8bb admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:09.431045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1c327-84d8-4390-9c17-ee69eb79e625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:09.431635 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1c327-84d8-4390-9c17-ee69eb79e625 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:09.432203 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1c327-84d8-4390-9c17-ee69eb79e625 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:09.432483 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1c327-84d8-4390-9c17-ee69eb79e625 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:09.433120 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1c327-84d8-4390-9c17-ee69eb79e625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:09.434676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1c327-84d8-4390-9c17-ee69eb79e625 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:09.445628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1210/2403] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:09.449809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592598d-732b-4ee2-aa9b-c7fba35c2a40 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:09.459677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4592598d-732b-4ee2-aa9b-c7fba35c2a40 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:09.464271 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47146b83-8a10-4054-a58b-c3e228b56921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:09.464911 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47146b83-8a10-4054-a58b-c3e228b56921 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:09.465189 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47146b83-8a10-4054-a58b-c3e228b56921 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:09.465457 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47146b83-8a10-4054-a58b-c3e228b56921 admin admin] PATH_INFO: `/v3/projects/4478a22855654f3cadc43857b1cac35c/users/71ba46ce04d0465d8a1c70152ac71d05/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:09.481216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47146b83-8a10-4054-a58b-c3e228b56921 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4478a22855654f3cadc43857b1cac35c, user_id=71ba46ce04d0465d8a1c70152ac71d05, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:09.484235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47146b83-8a10-4054-a58b-c3e228b56921 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:09.498361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1194/2404] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:58:09 2022] PUT /identity/v3/projects/4478a22855654f3cadc43857b1cac35c/users/71ba46ce04d0465d8a1c70152ac71d05/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:09.502361 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1c327-84d8-4390-9c17-ee69eb79e625 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:09.512563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1c327-84d8-4390-9c17-ee69eb79e625 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:09.568590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee18e2e9-33c7-44df-b7d8-175a985d1ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:09.569115 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18e2e9-33c7-44df-b7d8-175a985d1ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:09.569384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18e2e9-33c7-44df-b7d8-175a985d1ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:09.569649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee18e2e9-33c7-44df-b7d8-175a985d1ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:09.570273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18e2e9-33c7-44df-b7d8-175a985d1ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:09.571889 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee18e2e9-33c7-44df-b7d8-175a985d1ea9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:09.582008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1211/2405] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:09.586661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47146b83-8a10-4054-a58b-c3e228b56921 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:09.597800 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47146b83-8a10-4054-a58b-c3e228b56921 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:09.602806 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e65136-0c88-4f05-a006-7c13f702d33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:09.603454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e65136-0c88-4f05-a006-7c13f702d33c admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:09.603732 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e65136-0c88-4f05-a006-7c13f702d33c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:09.604030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e65136-0c88-4f05-a006-7c13f702d33c admin admin] PATH_INFO: `/v3/projects/4478a22855654f3cadc43857b1cac35c/users/71ba46ce04d0465d8a1c70152ac71d05/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:09.605927 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e65136-0c88-4f05-a006-7c13f702d33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4478a22855654f3cadc43857b1cac35c, user_id=71ba46ce04d0465d8a1c70152ac71d05, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:09.608630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e65136-0c88-4f05-a006-7c13f702d33c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:09.619747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1195/2406] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:58:09 2022] PUT /identity/v3/projects/4478a22855654f3cadc43857b1cac35c/users/71ba46ce04d0465d8a1c70152ac71d05/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:09.631398 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08a81a-a5b3-4b12-adb9-1bbddcd27a75 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:09.631719 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08a81a-a5b3-4b12-adb9-1bbddcd27a75 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:09.632038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08a81a-a5b3-4b12-adb9-1bbddcd27a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:09.660590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ca08a81a-a5b3-4b12-adb9-1bbddcd27a75 None None] MFA Rules not processed for user `71ba46ce04d0465d8a1c70152ac71d05`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:09.661819 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ca08a81a-a5b3-4b12-adb9-1bbddcd27a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:09.732073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1212/2407] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:09 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:58:09.743332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e65136-0c88-4f05-a006-7c13f702d33c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:09.745080 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-66e65136-0c88-4f05-a006-7c13f702d33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:09.803706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e65136-0c88-4f05-a006-7c13f702d33c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:09.807633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fd1af-ce6e-4a7f-92e3-f1f93cc87152 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:09.808267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fd1af-ce6e-4a7f-92e3-f1f93cc87152 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:09.808552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fd1af-ce6e-4a7f-92e3-f1f93cc87152 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:09.808824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fd1af-ce6e-4a7f-92e3-f1f93cc87152 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:09.818459 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fd1af-ce6e-4a7f-92e3-f1f93cc87152 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:09.819993 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fd1af-ce6e-4a7f-92e3-f1f93cc87152 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:09.829602 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1196/2408] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:09 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:09.909505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08a81a-a5b3-4b12-adb9-1bbddcd27a75 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:09.920160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08a81a-a5b3-4b12-adb9-1bbddcd27a75 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:09.924130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1664256-2271-4b85-a0c6-02106097aeed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:09.925262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1664256-2271-4b85-a0c6-02106097aeed service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:09.925543 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1664256-2271-4b85-a0c6-02106097aeed service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:09.925820 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1664256-2271-4b85-a0c6-02106097aeed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:09.932749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:09.990430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1664256-2271-4b85-a0c6-02106097aeed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:09.991860 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1664256-2271-4b85-a0c6-02106097aeed service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:10.000805 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1213/2409] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:09 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:12.973925 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979e236-0ef4-473d-b1e4-b1914defdd69 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:12.974409 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979e236-0ef4-473d-b1e4-b1914defdd69 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:12.974606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979e236-0ef4-473d-b1e4-b1914defdd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:12.994912 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c979e236-0ef4-473d-b1e4-b1914defdd69 None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:12.996151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c979e236-0ef4-473d-b1e4-b1914defdd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:13.028180 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:13.055317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1197/2410] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Sep 19 14:58:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:13.114685 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1664256-2271-4b85-a0c6-02106097aeed service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:13.116232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a1664256-2271-4b85-a0c6-02106097aeed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:13.195007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1664256-2271-4b85-a0c6-02106097aeed service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:13.200290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e85515-fd3e-4dcd-90d1-8e6492ca155f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:13.200943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e85515-fd3e-4dcd-90d1-8e6492ca155f service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:13.201223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e85515-fd3e-4dcd-90d1-8e6492ca155f service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:13.201498 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e85515-fd3e-4dcd-90d1-8e6492ca155f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:13.210961 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e85515-fd3e-4dcd-90d1-8e6492ca155f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:13.212588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e85515-fd3e-4dcd-90d1-8e6492ca155f service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:13.223104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1214/2411] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 14:58:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 14:58:14.172813 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979e236-0ef4-473d-b1e4-b1914defdd69 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:14.259236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979e236-0ef4-473d-b1e4-b1914defdd69 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:14.263743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcb7ef-7c35-463c-98d7-9734250a2237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:14.264437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcb7ef-7c35-463c-98d7-9734250a2237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:14.264716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcb7ef-7c35-463c-98d7-9734250a2237 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:14.265174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcb7ef-7c35-463c-98d7-9734250a2237 admin admin] PATH_INFO: `/v3/users/a3afb901f13c40fca50007ca536b49b2` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:14.267655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcb7ef-7c35-463c-98d7-9734250a2237 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3afb901f13c40fca50007ca536b49b2)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:14.271545 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcb7ef-7c35-463c-98d7-9734250a2237 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:14.327709 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f7fcb7ef-7c35-463c-98d7-9734250a2237 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3afb901f13c40fca50007ca536b49b2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:14.336964 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f7fcb7ef-7c35-463c-98d7-9734250a2237 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3afb901f13c40fca50007ca536b49b2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:14.350851 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f7fcb7ef-7c35-463c-98d7-9734250a2237 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3afb901f13c40fca50007ca536b49b2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:14.362175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f7fcb7ef-7c35-463c-98d7-9734250a2237 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3afb901f13c40fca50007ca536b49b2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:14.363137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-f7fcb7ef-7c35-463c-98d7-9734250a2237 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3afb901f13c40fca50007ca536b49b2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:14.370583 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1198/2412] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:58:14 2022] DELETE /identity/v3/users/a3afb901f13c40fca50007ca536b49b2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:14.376229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa7276-7f2f-4ed2-b09b-ddb950247e3f None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:14.376827 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa7276-7f2f-4ed2-b09b-ddb950247e3f None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:14.377193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa7276-7f2f-4ed2-b09b-ddb950247e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:14.405048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d7fa7276-7f2f-4ed2-b09b-ddb950247e3f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:14.406488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d7fa7276-7f2f-4ed2-b09b-ddb950247e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:14.477026 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1215/2413] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:14.487615 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcb7ef-7c35-463c-98d7-9734250a2237 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:14.489229 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f7fcb7ef-7c35-463c-98d7-9734250a2237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:14.558106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcb7ef-7c35-463c-98d7-9734250a2237 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:14.561905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860b8ee-745f-4e64-9d3f-d8284e105580 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:14.562533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860b8ee-745f-4e64-9d3f-d8284e105580 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:14.562812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860b8ee-745f-4e64-9d3f-d8284e105580 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:14.563084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860b8ee-745f-4e64-9d3f-d8284e105580 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:14.574963 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860b8ee-745f-4e64-9d3f-d8284e105580 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:14.576558 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860b8ee-745f-4e64-9d3f-d8284e105580 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:14.588737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1199/2414] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:58:14.979942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa7276-7f2f-4ed2-b09b-ddb950247e3f None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:14.979942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d7fa7276-7f2f-4ed2-b09b-ddb950247e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:15.015623 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fa7276-7f2f-4ed2-b09b-ddb950247e3f None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:15.034145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:15.034811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:15.035992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:15.036428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c admin admin] PATH_INFO: `/v3/projects/70c8cc1e4d9a4794ac587d52023bc1a3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:15.040549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c admin admin] RBAC: Authorizing `identity:delete_project(project_id=70c8cc1e4d9a4794ac587d52023bc1a3)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:15.042514 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:15.100580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:15.129553 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70c8cc1e4d9a4794ac587d52023bc1a3', 'request_id': 'req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:15.157533 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70c8cc1e4d9a4794ac587d52023bc1a3', 'request_id': 'req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:15.167699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70c8cc1e4d9a4794ac587d52023bc1a3', 'request_id': 'req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:15.198488 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c admin admin] The token cache is being invalidate because project 70c8cc1e4d9a4794ac587d52023bc1a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:58:15.198713 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:15.199476 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:15.206489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860b8ee-745f-4e64-9d3f-d8284e105580 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:15.213789 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1216/2415] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:58:14 2022] DELETE /identity/v3/projects/70c8cc1e4d9a4794ac587d52023bc1a3 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:15.214170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:15.227605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860b8ee-745f-4e64-9d3f-d8284e105580 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:15.325995 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250ce04-ab15-4bc7-b2d1-a948f8bb15f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:15.326564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250ce04-ab15-4bc7-b2d1-a948f8bb15f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:15.327369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250ce04-ab15-4bc7-b2d1-a948f8bb15f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:15.328364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250ce04-ab15-4bc7-b2d1-a948f8bb15f5 admin admin] PATH_INFO: `/v3/users/71ba46ce04d0465d8a1c70152ac71d05` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:15.329691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250ce04-ab15-4bc7-b2d1-a948f8bb15f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71ba46ce04d0465d8a1c70152ac71d05)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:15.333685 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250ce04-ab15-4bc7-b2d1-a948f8bb15f5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:15.367303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:15.369581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:15.433275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8250ce04-ab15-4bc7-b2d1-a948f8bb15f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71ba46ce04d0465d8a1c70152ac71d05'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:15.442280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8250ce04-ab15-4bc7-b2d1-a948f8bb15f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71ba46ce04d0465d8a1c70152ac71d05'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:15.458481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8250ce04-ab15-4bc7-b2d1-a948f8bb15f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71ba46ce04d0465d8a1c70152ac71d05'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:15.469277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8250ce04-ab15-4bc7-b2d1-a948f8bb15f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71ba46ce04d0465d8a1c70152ac71d05'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:15.470267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-8250ce04-ab15-4bc7-b2d1-a948f8bb15f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71ba46ce04d0465d8a1c70152ac71d05'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:15.479216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1200/2416] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:58:15 2022] DELETE /identity/v3/users/71ba46ce04d0465d8a1c70152ac71d05 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:15.505123 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd8fc7-1057-48ab-a5b4-dfa1ca41704c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:15.595108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad87519-3148-4277-ab01-6be3cf8d9134 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:15.595933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad87519-3148-4277-ab01-6be3cf8d9134 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:15.596236 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad87519-3148-4277-ab01-6be3cf8d9134 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:15.597434 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad87519-3148-4277-ab01-6be3cf8d9134 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:15.599746 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1ad87519-3148-4277-ab01-6be3cf8d9134 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:15.617874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad87519-3148-4277-ab01-6be3cf8d9134 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:15.620778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad87519-3148-4277-ab01-6be3cf8d9134 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:15.652743 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1217/2417] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:58:15.925049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250ce04-ab15-4bc7-b2d1-a948f8bb15f5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:15.926979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8250ce04-ab15-4bc7-b2d1-a948f8bb15f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:16.057346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250ce04-ab15-4bc7-b2d1-a948f8bb15f5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:16.061992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23f371f-1ba7-4db6-9da4-bbaa6b5706b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:16.062637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f371f-1ba7-4db6-9da4-bbaa6b5706b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:16.062940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f371f-1ba7-4db6-9da4-bbaa6b5706b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:16.063187 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f371f-1ba7-4db6-9da4-bbaa6b5706b4 admin admin] PATH_INFO: `/v3/projects/4478a22855654f3cadc43857b1cac35c` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:16.064379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f371f-1ba7-4db6-9da4-bbaa6b5706b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4478a22855654f3cadc43857b1cac35c)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:16.066889 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23f371f-1ba7-4db6-9da4-bbaa6b5706b4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:16.117902 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:16.166550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b23f371f-1ba7-4db6-9da4-bbaa6b5706b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4478a22855654f3cadc43857b1cac35c', 'request_id': 'req-b23f371f-1ba7-4db6-9da4-bbaa6b5706b4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:16.174875 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b23f371f-1ba7-4db6-9da4-bbaa6b5706b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4478a22855654f3cadc43857b1cac35c', 'request_id': 'req-b23f371f-1ba7-4db6-9da4-bbaa6b5706b4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:16.193254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b23f371f-1ba7-4db6-9da4-bbaa6b5706b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4478a22855654f3cadc43857b1cac35c', 'request_id': 'req-b23f371f-1ba7-4db6-9da4-bbaa6b5706b4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:16.219191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b23f371f-1ba7-4db6-9da4-bbaa6b5706b4 admin admin] The token cache is being invalidate because project 4478a22855654f3cadc43857b1cac35c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:58:16.219967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b23f371f-1ba7-4db6-9da4-bbaa6b5706b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:16.220255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b23f371f-1ba7-4db6-9da4-bbaa6b5706b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:16.222003 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1201/2418] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:58:15 2022] DELETE /identity/v3/projects/4478a22855654f3cadc43857b1cac35c => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:58:16.227182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af901593-7622-49ca-8930-d241baba7b93 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:16.227475 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af901593-7622-49ca-8930-d241baba7b93 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:16.227763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af901593-7622-49ca-8930-d241baba7b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:16.253829 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-af901593-7622-49ca-8930-d241baba7b93 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:16.255613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-af901593-7622-49ca-8930-d241baba7b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:16.316180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:16.340505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1218/2419] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:16.360036 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f28ea-6ce2-4f8e-9efb-d7aeabd651e2 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:16.360328 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f28ea-6ce2-4f8e-9efb-d7aeabd651e2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:16.360601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f28ea-6ce2-4f8e-9efb-d7aeabd651e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:16.383632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-271f28ea-6ce2-4f8e-9efb-d7aeabd651e2 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:16.384935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-271f28ea-6ce2-4f8e-9efb-d7aeabd651e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:16.393870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1202/2420] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:16.398236 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af901593-7622-49ca-8930-d241baba7b93 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:16.408970 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af901593-7622-49ca-8930-d241baba7b93 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:16.419577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3138f3-71f1-4970-b2fb-7a6cefdf1e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:16.420296 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3138f3-71f1-4970-b2fb-7a6cefdf1e80 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:16.420674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3138f3-71f1-4970-b2fb-7a6cefdf1e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:16.420964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3138f3-71f1-4970-b2fb-7a6cefdf1e80 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:16.421676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3138f3-71f1-4970-b2fb-7a6cefdf1e80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:58:16.421991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3138f3-71f1-4970-b2fb-7a6cefdf1e80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:16.423721 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3138f3-71f1-4970-b2fb-7a6cefdf1e80 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:16.439061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1219/2421] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:58:16.444432 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f28ea-6ce2-4f8e-9efb-d7aeabd651e2 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:16.458289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f28ea-6ce2-4f8e-9efb-d7aeabd651e2 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:16.463407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e9567-1688-4e5f-93ba-f31bb6828cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:16.464393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e9567-1688-4e5f-93ba-f31bb6828cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:16.465032 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e9567-1688-4e5f-93ba-f31bb6828cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:16.465747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382e9567-1688-4e5f-93ba-f31bb6828cb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:16.466540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e9567-1688-4e5f-93ba-f31bb6828cb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:16.469411 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382e9567-1688-4e5f-93ba-f31bb6828cb7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:16.507242 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1203/2422] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:58:16 2022] POST /identity/v3/projects => generated 381 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:16.511090 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3138f3-71f1-4970-b2fb-7a6cefdf1e80 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:16.522753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3138f3-71f1-4970-b2fb-7a6cefdf1e80 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:16.604593 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e198fc-0057-453a-9b6c-0a527fbed1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:16.606301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e198fc-0057-453a-9b6c-0a527fbed1de admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:16.607113 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e198fc-0057-453a-9b6c-0a527fbed1de admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:16.610125 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e198fc-0057-453a-9b6c-0a527fbed1de admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:16.611162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e198fc-0057-453a-9b6c-0a527fbed1de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:16.613511 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e198fc-0057-453a-9b6c-0a527fbed1de admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:16.642146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1220/2423] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:58:16 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:16.645858 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e9567-1688-4e5f-93ba-f31bb6828cb7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:16.665230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382e9567-1688-4e5f-93ba-f31bb6828cb7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:16.671159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa2d33-9aab-4f69-b383-73749faef45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:16.672123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa2d33-9aab-4f69-b383-73749faef45c admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:16.672618 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa2d33-9aab-4f69-b383-73749faef45c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:16.673093 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaa2d33-9aab-4f69-b383-73749faef45c admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:16.673948 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa2d33-9aab-4f69-b383-73749faef45c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:16.676324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaa2d33-9aab-4f69-b383-73749faef45c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:16.687737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1204/2424] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:16.692578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e198fc-0057-453a-9b6c-0a527fbed1de admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:16.705963 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e198fc-0057-453a-9b6c-0a527fbed1de admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:16.711558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4fa30-aeff-4212-80e1-750fea704b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:16.712487 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4fa30-aeff-4212-80e1-750fea704b8c admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:16.712951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4fa30-aeff-4212-80e1-750fea704b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:16.713232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4fa30-aeff-4212-80e1-750fea704b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:16.713846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4fa30-aeff-4212-80e1-750fea704b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:16.715496 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4fa30-aeff-4212-80e1-750fea704b8c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:16.728034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1221/2425] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:16.732619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa2d33-9aab-4f69-b383-73749faef45c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:16.745633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaa2d33-9aab-4f69-b383-73749faef45c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:16.753089 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88408693-5f14-4627-b1c7-6d8015c29a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:16.753705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88408693-5f14-4627-b1c7-6d8015c29a17 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:16.753961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88408693-5f14-4627-b1c7-6d8015c29a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:16.754306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88408693-5f14-4627-b1c7-6d8015c29a17 admin admin] PATH_INFO: `/v3/projects/64881ae3da084b31ade34087c16324a0/users/6e0f0eeb18b1409a9a6ee14139fc2746/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:16.774438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88408693-5f14-4627-b1c7-6d8015c29a17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64881ae3da084b31ade34087c16324a0, user_id=6e0f0eeb18b1409a9a6ee14139fc2746, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:16.779119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88408693-5f14-4627-b1c7-6d8015c29a17 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:16.794230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1205/2426] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:58:16 2022] PUT /identity/v3/projects/64881ae3da084b31ade34087c16324a0/users/6e0f0eeb18b1409a9a6ee14139fc2746/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:16.807726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d00559-085b-433a-ad0e-5e08ea424f60 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:16.808079 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d00559-085b-433a-ad0e-5e08ea424f60 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:16.808353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d00559-085b-433a-ad0e-5e08ea424f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:16.841705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-24d00559-085b-433a-ad0e-5e08ea424f60 None None] MFA Rules not processed for user `6e0f0eeb18b1409a9a6ee14139fc2746`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:16.842990 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-24d00559-085b-433a-ad0e-5e08ea424f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:16.926387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1222/2427] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:16.935375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88408693-5f14-4627-b1c7-6d8015c29a17 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:17.022788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88408693-5f14-4627-b1c7-6d8015c29a17 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:17.032961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b3fa2-767c-48ab-b633-a7a6bd76f9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:17.034051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b3fa2-767c-48ab-b633-a7a6bd76f9a5 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:17.034502 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b3fa2-767c-48ab-b633-a7a6bd76f9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:17.034944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b3fa2-767c-48ab-b633-a7a6bd76f9a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:17.035743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b3fa2-767c-48ab-b633-a7a6bd76f9a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:17.038010 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b3fa2-767c-48ab-b633-a7a6bd76f9a5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:17.074785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d00559-085b-433a-ad0e-5e08ea424f60 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:17.079980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1206/2428] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:58:16 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:17.086853 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d00559-085b-433a-ad0e-5e08ea424f60 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:17.142935 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:17.165840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22301110-1c36-4373-92e8-d6934263e4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:17.166833 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22301110-1c36-4373-92e8-d6934263e4e6 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:17.167353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22301110-1c36-4373-92e8-d6934263e4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:17.167802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22301110-1c36-4373-92e8-d6934263e4e6 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:17.168754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22301110-1c36-4373-92e8-d6934263e4e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:17.170994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22301110-1c36-4373-92e8-d6934263e4e6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:17.202783 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1223/2429] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:58:17 2022] POST /identity/v3/users => generated 296 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:17.203573 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b3fa2-767c-48ab-b633-a7a6bd76f9a5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:17.224750 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b3fa2-767c-48ab-b633-a7a6bd76f9a5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:17.230678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773e235-5ba3-41bf-83c6-58e3209e2b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:17.232065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773e235-5ba3-41bf-83c6-58e3209e2b79 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:17.232538 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773e235-5ba3-41bf-83c6-58e3209e2b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:17.232978 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773e235-5ba3-41bf-83c6-58e3209e2b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:17.234292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773e235-5ba3-41bf-83c6-58e3209e2b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:17.236417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773e235-5ba3-41bf-83c6-58e3209e2b79 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:17.252185 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1207/2430] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:17.256396 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22301110-1c36-4373-92e8-d6934263e4e6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:17.270347 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22301110-1c36-4373-92e8-d6934263e4e6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:17.275405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b90737-fa94-4e16-aaf7-f826c2488986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:17.276383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b90737-fa94-4e16-aaf7-f826c2488986 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:17.276661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b90737-fa94-4e16-aaf7-f826c2488986 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:17.276931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b90737-fa94-4e16-aaf7-f826c2488986 admin admin] PATH_INFO: `/v3/projects/79abf609a05d4d4285f25b1fde1f4a8c/users/9daa1a9b4bfb4bed9d158b66465a5f3c/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:17.316991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b90737-fa94-4e16-aaf7-f826c2488986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79abf609a05d4d4285f25b1fde1f4a8c, user_id=9daa1a9b4bfb4bed9d158b66465a5f3c, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:17.321873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b90737-fa94-4e16-aaf7-f826c2488986 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:17.344680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1224/2431] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:58:17 2022] PUT /identity/v3/projects/79abf609a05d4d4285f25b1fde1f4a8c/users/9daa1a9b4bfb4bed9d158b66465a5f3c/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:17.348843 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773e235-5ba3-41bf-83c6-58e3209e2b79 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:17.360940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773e235-5ba3-41bf-83c6-58e3209e2b79 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:17.396821 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:17.479785 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c428f81-4558-4fe0-811e-936ba8fc4a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:17.480267 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c428f81-4558-4fe0-811e-936ba8fc4a0e admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:17.482426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c428f81-4558-4fe0-811e-936ba8fc4a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:17.482702 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c428f81-4558-4fe0-811e-936ba8fc4a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:17.483352 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c428f81-4558-4fe0-811e-936ba8fc4a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:17.485207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c428f81-4558-4fe0-811e-936ba8fc4a0e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:17.497508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1208/2432] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:17 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:58:17.501795 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b90737-fa94-4e16-aaf7-f826c2488986 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:17.513962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b90737-fa94-4e16-aaf7-f826c2488986 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:17.520918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84173ba1-40ce-4bee-af44-ef20642276c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:17.522170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84173ba1-40ce-4bee-af44-ef20642276c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:17.522682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84173ba1-40ce-4bee-af44-ef20642276c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:17.523176 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84173ba1-40ce-4bee-af44-ef20642276c0 admin admin] PATH_INFO: `/v3/projects/79abf609a05d4d4285f25b1fde1f4a8c/users/9daa1a9b4bfb4bed9d158b66465a5f3c/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:17.525614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84173ba1-40ce-4bee-af44-ef20642276c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79abf609a05d4d4285f25b1fde1f4a8c, user_id=9daa1a9b4bfb4bed9d158b66465a5f3c, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:17.528640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84173ba1-40ce-4bee-af44-ef20642276c0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:17.551532 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1225/2433] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:58:17 2022] PUT /identity/v3/projects/79abf609a05d4d4285f25b1fde1f4a8c/users/9daa1a9b4bfb4bed9d158b66465a5f3c/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:17.553745 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a5abb-1831-491c-aa3f-3a21c446dd18 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:17.554396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a5abb-1831-491c-aa3f-3a21c446dd18 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:17.554961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a5abb-1831-491c-aa3f-3a21c446dd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:17.589946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b71a5abb-1831-491c-aa3f-3a21c446dd18 None None] MFA Rules not processed for user `9daa1a9b4bfb4bed9d158b66465a5f3c`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:17.591604 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b71a5abb-1831-491c-aa3f-3a21c446dd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:17.687861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1209/2434] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:17.700190 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84173ba1-40ce-4bee-af44-ef20642276c0 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:17.702975 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-84173ba1-40ce-4bee-af44-ef20642276c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:17.781531 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84173ba1-40ce-4bee-af44-ef20642276c0 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:17.786611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560f544-932c-489f-ac3a-e2a286e98866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:17.787485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560f544-932c-489f-ac3a-e2a286e98866 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:17.787984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560f544-932c-489f-ac3a-e2a286e98866 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:17.788257 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560f544-932c-489f-ac3a-e2a286e98866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:17.798221 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560f544-932c-489f-ac3a-e2a286e98866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:17.800322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560f544-932c-489f-ac3a-e2a286e98866 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:17.824034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1226/2435] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:17.909882 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a5abb-1831-491c-aa3f-3a21c446dd18 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:17.922971 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a5abb-1831-491c-aa3f-3a21c446dd18 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:17.928052 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f1687-223d-4f25-9e04-d5e271d0da3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:17.928945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f1687-223d-4f25-9e04-d5e271d0da3f service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:17.929409 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f1687-223d-4f25-9e04-d5e271d0da3f service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:17.929879 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f1687-223d-4f25-9e04-d5e271d0da3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:18.012997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f1687-223d-4f25-9e04-d5e271d0da3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:18.014795 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f1687-223d-4f25-9e04-d5e271d0da3f service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:18.028285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1210/2436] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:18.438012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560f544-932c-489f-ac3a-e2a286e98866 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:18.515787 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560f544-932c-489f-ac3a-e2a286e98866 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:18.521357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d5e87-1586-4274-9f52-944c449c11ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:18.522014 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d5e87-1586-4274-9f52-944c449c11ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:18.522396 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d5e87-1586-4274-9f52-944c449c11ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:18.522650 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8d5e87-1586-4274-9f52-944c449c11ab admin admin] PATH_INFO: `/v3/users/6e0f0eeb18b1409a9a6ee14139fc2746` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:18.523815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d5e87-1586-4274-9f52-944c449c11ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e0f0eeb18b1409a9a6ee14139fc2746)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:18.526051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8d5e87-1586-4274-9f52-944c449c11ab admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:18.581044 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:18.600539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9e8d5e87-1586-4274-9f52-944c449c11ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e0f0eeb18b1409a9a6ee14139fc2746'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:18.612597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9e8d5e87-1586-4274-9f52-944c449c11ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e0f0eeb18b1409a9a6ee14139fc2746'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:18.635550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9e8d5e87-1586-4274-9f52-944c449c11ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e0f0eeb18b1409a9a6ee14139fc2746'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:18.649262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9e8d5e87-1586-4274-9f52-944c449c11ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e0f0eeb18b1409a9a6ee14139fc2746'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:18.650419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-9e8d5e87-1586-4274-9f52-944c449c11ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e0f0eeb18b1409a9a6ee14139fc2746'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:18.658146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1227/2437] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:58:18 2022] DELETE /identity/v3/users/6e0f0eeb18b1409a9a6ee14139fc2746 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:18.663196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75547ab-21ae-43ea-9a42-b6423cbb6a84 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:18.663699 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75547ab-21ae-43ea-9a42-b6423cbb6a84 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:18.663808 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75547ab-21ae-43ea-9a42-b6423cbb6a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:18.691755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d75547ab-21ae-43ea-9a42-b6423cbb6a84 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:18.694278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d75547ab-21ae-43ea-9a42-b6423cbb6a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:18.762928 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:18.830459 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1211/2438] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:18.844104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d5e87-1586-4274-9f52-944c449c11ab admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:18.847219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9e8d5e87-1586-4274-9f52-944c449c11ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:19.002806 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8d5e87-1586-4274-9f52-944c449c11ab admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:19.007291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179eb46-a767-4721-8866-381390112510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:19.008196 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179eb46-a767-4721-8866-381390112510 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:19.008579 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179eb46-a767-4721-8866-381390112510 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:19.008921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179eb46-a767-4721-8866-381390112510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:19.023594 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179eb46-a767-4721-8866-381390112510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:19.025739 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179eb46-a767-4721-8866-381390112510 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:19.055997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1228/2439] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:19.528581 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75547ab-21ae-43ea-9a42-b6423cbb6a84 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:19.530444 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d75547ab-21ae-43ea-9a42-b6423cbb6a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:19.562555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75547ab-21ae-43ea-9a42-b6423cbb6a84 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:19.571531 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:19.572671 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:19.581867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:19.583456 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a admin admin] PATH_INFO: `/v3/projects/64881ae3da084b31ade34087c16324a0` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:19.587399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a admin admin] RBAC: Authorizing `identity:delete_project(project_id=64881ae3da084b31ade34087c16324a0)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:19.594086 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:19.718814 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64881ae3da084b31ade34087c16324a0', 'request_id': 'req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:19.744590 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64881ae3da084b31ade34087c16324a0', 'request_id': 'req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:19.765415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64881ae3da084b31ade34087c16324a0', 'request_id': 'req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:19.804160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a admin admin] The token cache is being invalidate because project 64881ae3da084b31ade34087c16324a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:58:19.806794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:19.807879 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:19.813723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1212/2440] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:58:19 2022] DELETE /identity/v3/projects/64881ae3da084b31ade34087c16324a0 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:19.816209 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179eb46-a767-4721-8866-381390112510 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:19.831508 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179eb46-a767-4721-8866-381390112510 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:19.915617 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa3d0e-c850-4fbb-b5fc-5761885a096c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:19.916578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa3d0e-c850-4fbb-b5fc-5761885a096c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:19.916866 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa3d0e-c850-4fbb-b5fc-5761885a096c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:19.917136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa3d0e-c850-4fbb-b5fc-5761885a096c admin admin] PATH_INFO: `/v3/users/9daa1a9b4bfb4bed9d158b66465a5f3c` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:19.920021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa3d0e-c850-4fbb-b5fc-5761885a096c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9daa1a9b4bfb4bed9d158b66465a5f3c)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:19.922120 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa3d0e-c850-4fbb-b5fc-5761885a096c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:20.009618 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-83aa3d0e-c850-4fbb-b5fc-5761885a096c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9daa1a9b4bfb4bed9d158b66465a5f3c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:20.027686 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-83aa3d0e-c850-4fbb-b5fc-5761885a096c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9daa1a9b4bfb4bed9d158b66465a5f3c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:20.031951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:20.062889 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-83aa3d0e-c850-4fbb-b5fc-5761885a096c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9daa1a9b4bfb4bed9d158b66465a5f3c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:20.074219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-83aa3d0e-c850-4fbb-b5fc-5761885a096c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9daa1a9b4bfb4bed9d158b66465a5f3c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:20.075366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-83aa3d0e-c850-4fbb-b5fc-5761885a096c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9daa1a9b4bfb4bed9d158b66465a5f3c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:20.084294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1229/2441] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:58:19 2022] DELETE /identity/v3/users/9daa1a9b4bfb4bed9d158b66465a5f3c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:20.502072 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:20.504629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:20.538040 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba38caac-e00f-4b82-9c7c-b22f22b9a40a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:20.606067 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead2f33-4070-42ec-9015-1d59a50afb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:20.606821 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead2f33-4070-42ec-9015-1d59a50afb08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:20.607160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead2f33-4070-42ec-9015-1d59a50afb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:20.607498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead2f33-4070-42ec-9015-1d59a50afb08 admin admin] PATH_INFO: `/v3/projects/79abf609a05d4d4285f25b1fde1f4a8c` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:20.608897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead2f33-4070-42ec-9015-1d59a50afb08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79abf609a05d4d4285f25b1fde1f4a8c)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:20.610990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead2f33-4070-42ec-9015-1d59a50afb08 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:20.665179 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-9ead2f33-4070-42ec-9015-1d59a50afb08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79abf609a05d4d4285f25b1fde1f4a8c', 'request_id': 'req-9ead2f33-4070-42ec-9015-1d59a50afb08'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:20.675071 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-9ead2f33-4070-42ec-9015-1d59a50afb08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79abf609a05d4d4285f25b1fde1f4a8c', 'request_id': 'req-9ead2f33-4070-42ec-9015-1d59a50afb08'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:20.692276 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-9ead2f33-4070-42ec-9015-1d59a50afb08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79abf609a05d4d4285f25b1fde1f4a8c', 'request_id': 'req-9ead2f33-4070-42ec-9015-1d59a50afb08'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:20.700047 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:20.714074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-9ead2f33-4070-42ec-9015-1d59a50afb08 admin admin] The token cache is being invalidate because project 79abf609a05d4d4285f25b1fde1f4a8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:58:20.714074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-9ead2f33-4070-42ec-9015-1d59a50afb08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:20.715686 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-9ead2f33-4070-42ec-9015-1d59a50afb08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:20.720076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1213/2442] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:58:20 2022] DELETE /identity/v3/projects/79abf609a05d4d4285f25b1fde1f4a8c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:20.724960 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dd175-ebf0-4125-8e46-80b8ff54ce33 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:20.725253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dd175-ebf0-4125-8e46-80b8ff54ce33 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:20.725577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dd175-ebf0-4125-8e46-80b8ff54ce33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:20.747561 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-826dd175-ebf0-4125-8e46-80b8ff54ce33 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:20.755428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-826dd175-ebf0-4125-8e46-80b8ff54ce33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:20.852086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1230/2443] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:20.859691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd414c07-f78a-42aa-95d1-f9d4b1a5eede None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:20.860505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd414c07-f78a-42aa-95d1-f9d4b1a5eede None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:20.860986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd414c07-f78a-42aa-95d1-f9d4b1a5eede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:20.883714 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-fd414c07-f78a-42aa-95d1-f9d4b1a5eede None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:20.885248 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-fd414c07-f78a-42aa-95d1-f9d4b1a5eede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:20.894988 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1214/2444] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:20.899308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dd175-ebf0-4125-8e46-80b8ff54ce33 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:20.910262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826dd175-ebf0-4125-8e46-80b8ff54ce33 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:20.916170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae605e3-930f-41d8-b5e7-f74cb251d6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:20.917046 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae605e3-930f-41d8-b5e7-f74cb251d6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:20.917574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae605e3-930f-41d8-b5e7-f74cb251d6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:20.917998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae605e3-930f-41d8-b5e7-f74cb251d6c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:20.918800 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae605e3-930f-41d8-b5e7-f74cb251d6c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:58:20.919250 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae605e3-930f-41d8-b5e7-f74cb251d6c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:20.921173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae605e3-930f-41d8-b5e7-f74cb251d6c0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:20.940136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1231/2445] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:58:20.945049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd414c07-f78a-42aa-95d1-f9d4b1a5eede None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:20.957649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd414c07-f78a-42aa-95d1-f9d4b1a5eede None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:20.963763 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d96798-b6af-4aa9-a6e2-2f86f454d4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:20.964061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d96798-b6af-4aa9-a6e2-2f86f454d4ee admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:20.964335 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d96798-b6af-4aa9-a6e2-2f86f454d4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:20.964708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d96798-b6af-4aa9-a6e2-2f86f454d4ee admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:20.965392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d96798-b6af-4aa9-a6e2-2f86f454d4ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:20.967416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d96798-b6af-4aa9-a6e2-2f86f454d4ee admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:21.013659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1215/2446] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:58:20 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:58:21.015393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae605e3-930f-41d8-b5e7-f74cb251d6c0 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:21.030317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae605e3-930f-41d8-b5e7-f74cb251d6c0 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:21.112740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d0508-9043-4885-9eb8-dba3d5a046dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:21.112891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d0508-9043-4885-9eb8-dba3d5a046dc admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:21.113176 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d0508-9043-4885-9eb8-dba3d5a046dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:21.113450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d0508-9043-4885-9eb8-dba3d5a046dc admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:21.114091 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d0508-9043-4885-9eb8-dba3d5a046dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:21.116040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d0508-9043-4885-9eb8-dba3d5a046dc admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:21.144206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1232/2447] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:58:21 2022] POST /identity/v3/users => generated 287 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:21.148177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d96798-b6af-4aa9-a6e2-2f86f454d4ee admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:21.160811 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d96798-b6af-4aa9-a6e2-2f86f454d4ee admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:21.168376 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb25ad-8739-495d-872f-5016553ef922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:21.169499 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb25ad-8739-495d-872f-5016553ef922 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:21.169499 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb25ad-8739-495d-872f-5016553ef922 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:21.170086 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfb25ad-8739-495d-872f-5016553ef922 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:21.170913 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb25ad-8739-495d-872f-5016553ef922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:21.172788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfb25ad-8739-495d-872f-5016553ef922 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:21.192092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1216/2448] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:58:21.193424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d0508-9043-4885-9eb8-dba3d5a046dc admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:21.204519 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d0508-9043-4885-9eb8-dba3d5a046dc admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:21.210135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e00348-35d8-4c33-ae3e-6e7075212d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:21.211079 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e00348-35d8-4c33-ae3e-6e7075212d7b admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:21.211561 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e00348-35d8-4c33-ae3e-6e7075212d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:21.213862 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e00348-35d8-4c33-ae3e-6e7075212d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:21.214826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e00348-35d8-4c33-ae3e-6e7075212d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:21.217151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e00348-35d8-4c33-ae3e-6e7075212d7b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:21.234582 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1233/2449] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:21.235556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb25ad-8739-495d-872f-5016553ef922 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:21.252911 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfb25ad-8739-495d-872f-5016553ef922 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:21.262488 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48d7b8-7178-42e7-84e7-d475e43221bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:21.275798 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48d7b8-7178-42e7-84e7-d475e43221bd admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:21.276112 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48d7b8-7178-42e7-84e7-d475e43221bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:21.276552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48d7b8-7178-42e7-84e7-d475e43221bd admin admin] PATH_INFO: `/v3/projects/b77387d2f6964a16b040c54402625d1d/users/6a79dfb11aba4bf08b4b30759117e058/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:21.296149 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48d7b8-7178-42e7-84e7-d475e43221bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b77387d2f6964a16b040c54402625d1d, user_id=6a79dfb11aba4bf08b4b30759117e058, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:21.299735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48d7b8-7178-42e7-84e7-d475e43221bd admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:21.320395 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1217/2450] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:58:21 2022] PUT /identity/v3/projects/b77387d2f6964a16b040c54402625d1d/users/6a79dfb11aba4bf08b4b30759117e058/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:58:21.332745 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e68ff7-14e8-491c-9c88-722554ec012a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:21.333350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e68ff7-14e8-491c-9c88-722554ec012a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:21.333818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e68ff7-14e8-491c-9c88-722554ec012a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:21.375419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-00e68ff7-14e8-491c-9c88-722554ec012a None None] MFA Rules not processed for user `6a79dfb11aba4bf08b4b30759117e058`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:21.377815 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-00e68ff7-14e8-491c-9c88-722554ec012a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:21.419574 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:21.504011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1234/2451] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:21.519488 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48d7b8-7178-42e7-84e7-d475e43221bd admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:21.521722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-fb48d7b8-7178-42e7-84e7-d475e43221bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:21.613470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48d7b8-7178-42e7-84e7-d475e43221bd admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:21.618788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55eebdf-16d0-4d3c-8feb-df56c11e17ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:21.619413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55eebdf-16d0-4d3c-8feb-df56c11e17ad service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:21.619697 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55eebdf-16d0-4d3c-8feb-df56c11e17ad service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:21.620182 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55eebdf-16d0-4d3c-8feb-df56c11e17ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:21.632689 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55eebdf-16d0-4d3c-8feb-df56c11e17ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:21.634117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55eebdf-16d0-4d3c-8feb-df56c11e17ad service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:21.653618 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1218/2452] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:21.677785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e68ff7-14e8-491c-9c88-722554ec012a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:21.762143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e68ff7-14e8-491c-9c88-722554ec012a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:21.768982 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde748f6-c0e4-49ec-b08c-6227b4cf2fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:21.770111 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde748f6-c0e4-49ec-b08c-6227b4cf2fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:21.770651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde748f6-c0e4-49ec-b08c-6227b4cf2fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:21.771152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde748f6-c0e4-49ec-b08c-6227b4cf2fb8 admin admin] PATH_INFO: `/v3/users/6a79dfb11aba4bf08b4b30759117e058` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:21.772723 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde748f6-c0e4-49ec-b08c-6227b4cf2fb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a79dfb11aba4bf08b4b30759117e058)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:21.775273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde748f6-c0e4-49ec-b08c-6227b4cf2fb8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:21.876823 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fde748f6-c0e4-49ec-b08c-6227b4cf2fb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a79dfb11aba4bf08b4b30759117e058'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:21.889021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fde748f6-c0e4-49ec-b08c-6227b4cf2fb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a79dfb11aba4bf08b4b30759117e058'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:21.915978 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fde748f6-c0e4-49ec-b08c-6227b4cf2fb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a79dfb11aba4bf08b4b30759117e058'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:21.927090 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fde748f6-c0e4-49ec-b08c-6227b4cf2fb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a79dfb11aba4bf08b4b30759117e058'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:21.929224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-fde748f6-c0e4-49ec-b08c-6227b4cf2fb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a79dfb11aba4bf08b4b30759117e058'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:21.940930 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1235/2453] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:58:21 2022] DELETE /identity/v3/users/6a79dfb11aba4bf08b4b30759117e058 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:21.945512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212e81c-49dc-4136-b410-f2971636eb05 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:21.946057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212e81c-49dc-4136-b410-f2971636eb05 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:21.946057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212e81c-49dc-4136-b410-f2971636eb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:21.974580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c212e81c-49dc-4136-b410-f2971636eb05 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:21.976076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c212e81c-49dc-4136-b410-f2971636eb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:22.070003 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1219/2454] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:22.097935 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde748f6-c0e4-49ec-b08c-6227b4cf2fb8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:22.099686 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fde748f6-c0e4-49ec-b08c-6227b4cf2fb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:22.198483 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde748f6-c0e4-49ec-b08c-6227b4cf2fb8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:22.203289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6edb1-821f-4719-9a77-4cd5b9a2b116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:22.204145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6edb1-821f-4719-9a77-4cd5b9a2b116 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:22.204499 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6edb1-821f-4719-9a77-4cd5b9a2b116 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:22.204841 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6edb1-821f-4719-9a77-4cd5b9a2b116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:22.217288 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:22.224749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6edb1-821f-4719-9a77-4cd5b9a2b116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:22.226938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6edb1-821f-4719-9a77-4cd5b9a2b116 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:22.242150 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1236/2455] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:22.902358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212e81c-49dc-4136-b410-f2971636eb05 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:22.903814 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c212e81c-49dc-4136-b410-f2971636eb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:22.937281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212e81c-49dc-4136-b410-f2971636eb05 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:22.944824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6c72f-462d-4cd7-8057-fc0a2d391f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:22.945562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6c72f-462d-4cd7-8057-fc0a2d391f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:22.946132 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6c72f-462d-4cd7-8057-fc0a2d391f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:22.946246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6c72f-462d-4cd7-8057-fc0a2d391f7f admin admin] PATH_INFO: `/v3/projects/b77387d2f6964a16b040c54402625d1d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:22.947448 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6c72f-462d-4cd7-8057-fc0a2d391f7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b77387d2f6964a16b040c54402625d1d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:22.949474 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6c72f-462d-4cd7-8057-fc0a2d391f7f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:22.998809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:23.024049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5fc6c72f-462d-4cd7-8057-fc0a2d391f7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b77387d2f6964a16b040c54402625d1d', 'request_id': 'req-5fc6c72f-462d-4cd7-8057-fc0a2d391f7f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:23.048106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5fc6c72f-462d-4cd7-8057-fc0a2d391f7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b77387d2f6964a16b040c54402625d1d', 'request_id': 'req-5fc6c72f-462d-4cd7-8057-fc0a2d391f7f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:23.064197 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5fc6c72f-462d-4cd7-8057-fc0a2d391f7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b77387d2f6964a16b040c54402625d1d', 'request_id': 'req-5fc6c72f-462d-4cd7-8057-fc0a2d391f7f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:23.095692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5fc6c72f-462d-4cd7-8057-fc0a2d391f7f admin admin] The token cache is being invalidate because project b77387d2f6964a16b040c54402625d1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:58:23.096429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5fc6c72f-462d-4cd7-8057-fc0a2d391f7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:23.097241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5fc6c72f-462d-4cd7-8057-fc0a2d391f7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:23.099306 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1220/2456] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:58:22 2022] DELETE /identity/v3/projects/b77387d2f6964a16b040c54402625d1d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:23.105413 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580d14e-371a-45d3-aa99-a9da65b46797 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:23.106211 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580d14e-371a-45d3-aa99-a9da65b46797 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:23.106211 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580d14e-371a-45d3-aa99-a9da65b46797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:23.130616 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2580d14e-371a-45d3-aa99-a9da65b46797 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:23.131872 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2580d14e-371a-45d3-aa99-a9da65b46797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:23.230349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1237/2457] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:23.232432 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71224eeb-6f7a-404d-8c5f-1be5901ac37b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:23.232432 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71224eeb-6f7a-404d-8c5f-1be5901ac37b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:23.232763 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71224eeb-6f7a-404d-8c5f-1be5901ac37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:23.261396 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-71224eeb-6f7a-404d-8c5f-1be5901ac37b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:23.263299 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-71224eeb-6f7a-404d-8c5f-1be5901ac37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:23.277919 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1221/2458] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:23.278869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580d14e-371a-45d3-aa99-a9da65b46797 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:23.290037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580d14e-371a-45d3-aa99-a9da65b46797 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:23.297575 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062a927-9d58-49e9-8fea-24c108a8be65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:23.298311 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062a927-9d58-49e9-8fea-24c108a8be65 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:23.299166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062a927-9d58-49e9-8fea-24c108a8be65 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:23.299546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062a927-9d58-49e9-8fea-24c108a8be65 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:23.301074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062a927-9d58-49e9-8fea-24c108a8be65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:58:23.301597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062a927-9d58-49e9-8fea-24c108a8be65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:23.306465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062a927-9d58-49e9-8fea-24c108a8be65 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:23.324576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1238/2459] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:58:23.325371 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff067d-c0cd-4cf7-b675-58e4f988c796 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:23.325525 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062a927-9d58-49e9-8fea-24c108a8be65 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:23.326110 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff067d-c0cd-4cf7-b675-58e4f988c796 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:23.326659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff067d-c0cd-4cf7-b675-58e4f988c796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:23.338972 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062a927-9d58-49e9-8fea-24c108a8be65 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:23.346136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2806dad-5520-4f01-95fd-108c0caea3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:23.346387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2806dad-5520-4f01-95fd-108c0caea3f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:23.347019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2806dad-5520-4f01-95fd-108c0caea3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:23.347505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2806dad-5520-4f01-95fd-108c0caea3f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:23.348790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2806dad-5520-4f01-95fd-108c0caea3f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:23.352424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2806dad-5520-4f01-95fd-108c0caea3f4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:23.365317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e6ff067d-c0cd-4cf7-b675-58e4f988c796 None None] MFA Rules not processed for user `73ef9d62f31d4135b6f079a26d791ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:23.365777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e6ff067d-c0cd-4cf7-b675-58e4f988c796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:23.389722 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1239/2460] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:58:23 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:23.395659 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2806dad-5520-4f01-95fd-108c0caea3f4 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:23.473307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1222/2461] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:58:23.515231 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2806dad-5520-4f01-95fd-108c0caea3f4 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:23.522711 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a78a11-38ab-4599-9341-0deefbc87a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:23.523418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a78a11-38ab-4599-9341-0deefbc87a9c admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:23.523717 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a78a11-38ab-4599-9341-0deefbc87a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:23.524184 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a78a11-38ab-4599-9341-0deefbc87a9c admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:23.525143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a78a11-38ab-4599-9341-0deefbc87a9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:23.528654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a78a11-38ab-4599-9341-0deefbc87a9c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:23.560083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1240/2462] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:58:23 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:23.564236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff067d-c0cd-4cf7-b675-58e4f988c796 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:23.576741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff067d-c0cd-4cf7-b675-58e4f988c796 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:23.582401 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a712dd6-96cd-4399-9821-ad87765be414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:23.583700 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a712dd6-96cd-4399-9821-ad87765be414 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:23.584021 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a712dd6-96cd-4399-9821-ad87765be414 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:23.584298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a712dd6-96cd-4399-9821-ad87765be414 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:23.584946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a712dd6-96cd-4399-9821-ad87765be414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:23.586562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a712dd6-96cd-4399-9821-ad87765be414 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:23.603772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1223/2463] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:58:23.604985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a78a11-38ab-4599-9341-0deefbc87a9c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:23.621882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a78a11-38ab-4599-9341-0deefbc87a9c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:23.636585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972f029-af78-46fb-83ed-729396a6f4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:23.637723 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972f029-af78-46fb-83ed-729396a6f4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:23.638077 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972f029-af78-46fb-83ed-729396a6f4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:23.638444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972f029-af78-46fb-83ed-729396a6f4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:23.639215 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972f029-af78-46fb-83ed-729396a6f4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:23.641126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972f029-af78-46fb-83ed-729396a6f4e4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:23.656902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1241/2464] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:23.661950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a712dd6-96cd-4399-9821-ad87765be414 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:23.673032 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a712dd6-96cd-4399-9821-ad87765be414 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:23.685711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f6794-c081-4107-964e-8b0f7d8246d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:23.686577 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f6794-c081-4107-964e-8b0f7d8246d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:23.687112 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f6794-c081-4107-964e-8b0f7d8246d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:23.687491 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f6794-c081-4107-964e-8b0f7d8246d8 admin admin] PATH_INFO: `/v3/projects/d40aaaba97d14586ade17642f9444b03/users/6062b03fcfad42b9bd07df0cd42e4f30/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:23.712750 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f6794-c081-4107-964e-8b0f7d8246d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d40aaaba97d14586ade17642f9444b03, user_id=6062b03fcfad42b9bd07df0cd42e4f30, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:23.715708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f6794-c081-4107-964e-8b0f7d8246d8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:23.740145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1224/2465] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:58:23 2022] PUT /identity/v3/projects/d40aaaba97d14586ade17642f9444b03/users/6062b03fcfad42b9bd07df0cd42e4f30/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:23.750852 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51381a5c-f66d-49bb-a822-87261e609192 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:23.752414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51381a5c-f66d-49bb-a822-87261e609192 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:23.753010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51381a5c-f66d-49bb-a822-87261e609192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:23.796437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-51381a5c-f66d-49bb-a822-87261e609192 None None] MFA Rules not processed for user `6062b03fcfad42b9bd07df0cd42e4f30`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:23.797862 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-51381a5c-f66d-49bb-a822-87261e609192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:23.836637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:23.910943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1242/2466] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:23.917126 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f6794-c081-4107-964e-8b0f7d8246d8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:23.993263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f6794-c081-4107-964e-8b0f7d8246d8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:23.999920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfa391-85a9-48cc-815f-4531632eecc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:24.000561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfa391-85a9-48cc-815f-4531632eecc9 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:24.000908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfa391-85a9-48cc-815f-4531632eecc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:24.001173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfa391-85a9-48cc-815f-4531632eecc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:24.001816 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cfa391-85a9-48cc-815f-4531632eecc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:24.003875 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cfa391-85a9-48cc-815f-4531632eecc9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:24.049584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1225/2467] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:58:23 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:24.050766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51381a5c-f66d-49bb-a822-87261e609192 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:24.080901 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51381a5c-f66d-49bb-a822-87261e609192 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:24.175091 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3322b-c928-4170-9162-f6315c0c54ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:24.175760 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3322b-c928-4170-9162-f6315c0c54ad admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:24.176032 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3322b-c928-4170-9162-f6315c0c54ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:24.176304 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3322b-c928-4170-9162-f6315c0c54ad admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:24.176945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3322b-c928-4170-9162-f6315c0c54ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:24.179890 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3322b-c928-4170-9162-f6315c0c54ad admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:24.210240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1243/2468] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:58:24 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:24.211512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfa391-85a9-48cc-815f-4531632eecc9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:24.223938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfa391-85a9-48cc-815f-4531632eecc9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:24.229522 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d61db2-80cb-4b80-8b0d-51cef04737bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:24.230150 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d61db2-80cb-4b80-8b0d-51cef04737bc admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:24.230461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d61db2-80cb-4b80-8b0d-51cef04737bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:24.230741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d61db2-80cb-4b80-8b0d-51cef04737bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:24.231627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d61db2-80cb-4b80-8b0d-51cef04737bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:24.233677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d61db2-80cb-4b80-8b0d-51cef04737bc admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:24.236130 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:24.264120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1226/2469] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:24.269817 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3322b-c928-4170-9162-f6315c0c54ad admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:24.285692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3322b-c928-4170-9162-f6315c0c54ad admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:24.291951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ebf71-ce16-4633-b423-ecae15175c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:24.292947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ebf71-ce16-4633-b423-ecae15175c35 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:24.293857 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ebf71-ce16-4633-b423-ecae15175c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:24.294291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ebf71-ce16-4633-b423-ecae15175c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:24.295391 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ebf71-ce16-4633-b423-ecae15175c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:24.316689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ebf71-ce16-4633-b423-ecae15175c35 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:24.333560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1244/2470] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:24.334254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d61db2-80cb-4b80-8b0d-51cef04737bc admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:24.346123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d61db2-80cb-4b80-8b0d-51cef04737bc admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:24.364836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91e981-3bb1-48d9-84b6-46e52a1e1cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:24.365898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91e981-3bb1-48d9-84b6-46e52a1e1cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:24.366332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91e981-3bb1-48d9-84b6-46e52a1e1cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:24.366613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91e981-3bb1-48d9-84b6-46e52a1e1cc0 admin admin] PATH_INFO: `/v3/projects/ecc5df3b30e74353b2064020a051b9ca/users/1ffcbce8487045458d9ebd34af9163a6/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:24.388663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91e981-3bb1-48d9-84b6-46e52a1e1cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecc5df3b30e74353b2064020a051b9ca, user_id=1ffcbce8487045458d9ebd34af9163a6, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:24.391350 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91e981-3bb1-48d9-84b6-46e52a1e1cc0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:24.412943 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1227/2471] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:58:24 2022] PUT /identity/v3/projects/ecc5df3b30e74353b2064020a051b9ca/users/1ffcbce8487045458d9ebd34af9163a6/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:24.490630 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d3e82-a707-45ce-bbaa-ab86eb80a0c1 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:24.490984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d3e82-a707-45ce-bbaa-ab86eb80a0c1 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:24.491189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d3e82-a707-45ce-bbaa-ab86eb80a0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:24.526052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-960d3e82-a707-45ce-bbaa-ab86eb80a0c1 None None] MFA Rules not processed for user `1ffcbce8487045458d9ebd34af9163a6`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:24.528163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-960d3e82-a707-45ce-bbaa-ab86eb80a0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:24.618992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1245/2472] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:58:24.624011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91e981-3bb1-48d9-84b6-46e52a1e1cc0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:24.709705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91e981-3bb1-48d9-84b6-46e52a1e1cc0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:24.715420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a6dff-4155-4d9a-8c2a-5fac2edb71ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:24.716300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a6dff-4155-4d9a-8c2a-5fac2edb71ea admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:24.716819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a6dff-4155-4d9a-8c2a-5fac2edb71ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:24.717298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a6dff-4155-4d9a-8c2a-5fac2edb71ea admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:24.718381 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a6dff-4155-4d9a-8c2a-5fac2edb71ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:24.721980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a6dff-4155-4d9a-8c2a-5fac2edb71ea admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:24.759995 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1228/2473] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:58:24 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:58:24.761828 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d3e82-a707-45ce-bbaa-ab86eb80a0c1 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:24.777814 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d3e82-a707-45ce-bbaa-ab86eb80a0c1 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:24.824506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:24.861707 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d3dbd-9e5d-432a-a6eb-850d5c16dfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:24.862717 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d3dbd-9e5d-432a-a6eb-850d5c16dfab admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:24.862993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d3dbd-9e5d-432a-a6eb-850d5c16dfab admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:24.863350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d3dbd-9e5d-432a-a6eb-850d5c16dfab admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:24.864231 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d3dbd-9e5d-432a-a6eb-850d5c16dfab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:24.866653 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d3dbd-9e5d-432a-a6eb-850d5c16dfab admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:24.903870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1246/2474] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:58:24 2022] POST /identity/v3/users => generated 298 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:58:24.904145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a6dff-4155-4d9a-8c2a-5fac2edb71ea admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:24.913956 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a6dff-4155-4d9a-8c2a-5fac2edb71ea admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:24.919506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018de6c-1633-40ff-b830-a27543cb7c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:24.920598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018de6c-1633-40ff-b830-a27543cb7c59 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:24.920988 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018de6c-1633-40ff-b830-a27543cb7c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:24.922474 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018de6c-1633-40ff-b830-a27543cb7c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:24.923319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018de6c-1633-40ff-b830-a27543cb7c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:24.926042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018de6c-1633-40ff-b830-a27543cb7c59 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:24.946332 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1229/2475] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:24 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:24.949540 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d3dbd-9e5d-432a-a6eb-850d5c16dfab admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:24.963687 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d3dbd-9e5d-432a-a6eb-850d5c16dfab admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:24.971259 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af4702-49fe-4f7f-93c0-7d043bbbc522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:24.972020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af4702-49fe-4f7f-93c0-7d043bbbc522 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:24.972307 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af4702-49fe-4f7f-93c0-7d043bbbc522 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:24.972584 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24af4702-49fe-4f7f-93c0-7d043bbbc522 admin admin] PATH_INFO: `/v3/projects/70a84a2cb3bc455ba1a1d90cd5913a0e/users/8352c3a3800d40a5a221c6803194fe61/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:24.991355 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af4702-49fe-4f7f-93c0-7d043bbbc522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a84a2cb3bc455ba1a1d90cd5913a0e, user_id=8352c3a3800d40a5a221c6803194fe61, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:24.995151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24af4702-49fe-4f7f-93c0-7d043bbbc522 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:25.011407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1247/2476] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:58:24 2022] PUT /identity/v3/projects/70a84a2cb3bc455ba1a1d90cd5913a0e/users/8352c3a3800d40a5a221c6803194fe61/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:25.015350 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018de6c-1633-40ff-b830-a27543cb7c59 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:25.030496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018de6c-1633-40ff-b830-a27543cb7c59 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:25.096898 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b84766-b318-4590-b01f-b01460d1feef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:25.097522 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b84766-b318-4590-b01f-b01460d1feef admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:25.097803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b84766-b318-4590-b01f-b01460d1feef admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:25.098212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b84766-b318-4590-b01f-b01460d1feef admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:25.099017 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b84766-b318-4590-b01f-b01460d1feef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:25.100656 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b84766-b318-4590-b01f-b01460d1feef admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:25.111696 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1230/2477] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:58:25 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:58:25.116065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af4702-49fe-4f7f-93c0-7d043bbbc522 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:25.128382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24af4702-49fe-4f7f-93c0-7d043bbbc522 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:25.134851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e1e04-8d38-4c8a-93e5-0c1336bb8454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:25.135604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e1e04-8d38-4c8a-93e5-0c1336bb8454 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:25.136010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e1e04-8d38-4c8a-93e5-0c1336bb8454 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:25.136356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e1e04-8d38-4c8a-93e5-0c1336bb8454 admin admin] PATH_INFO: `/v3/projects/70a84a2cb3bc455ba1a1d90cd5913a0e/users/8352c3a3800d40a5a221c6803194fe61/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:25.138397 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e1e04-8d38-4c8a-93e5-0c1336bb8454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a84a2cb3bc455ba1a1d90cd5913a0e, user_id=8352c3a3800d40a5a221c6803194fe61, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:25.141112 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e1e04-8d38-4c8a-93e5-0c1336bb8454 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:25.152529 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1248/2478] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:58:25 2022] PUT /identity/v3/projects/70a84a2cb3bc455ba1a1d90cd5913a0e/users/8352c3a3800d40a5a221c6803194fe61/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:25.163145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dca51e-d9e5-4b0c-891d-afea9f64d42d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:25.163435 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dca51e-d9e5-4b0c-891d-afea9f64d42d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:25.163734 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dca51e-d9e5-4b0c-891d-afea9f64d42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:25.195959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-11dca51e-d9e5-4b0c-891d-afea9f64d42d None None] MFA Rules not processed for user `8352c3a3800d40a5a221c6803194fe61`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:25.197317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-11dca51e-d9e5-4b0c-891d-afea9f64d42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:25.232585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:25.277478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1231/2479] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:25 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:25.290236 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e1e04-8d38-4c8a-93e5-0c1336bb8454 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:25.292040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-654e1e04-8d38-4c8a-93e5-0c1336bb8454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:25.351702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e1e04-8d38-4c8a-93e5-0c1336bb8454 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:25.355871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733aaee-5cf1-449d-b1c4-881529ec1407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:25.356545 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733aaee-5cf1-449d-b1c4-881529ec1407 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:25.356829 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733aaee-5cf1-449d-b1c4-881529ec1407 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:25.357104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733aaee-5cf1-449d-b1c4-881529ec1407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:25.413958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733aaee-5cf1-449d-b1c4-881529ec1407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:25.415455 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733aaee-5cf1-449d-b1c4-881529ec1407 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:25.424559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1249/2480] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:26.986985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dca51e-d9e5-4b0c-891d-afea9f64d42d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:26.988455 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-11dca51e-d9e5-4b0c-891d-afea9f64d42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:27.000534 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dca51e-d9e5-4b0c-891d-afea9f64d42d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:27.004688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ca10a-11c0-450c-a4c0-62103f11624d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:27.005314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ca10a-11c0-450c-a4c0-62103f11624d service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:27.005603 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ca10a-11c0-450c-a4c0-62103f11624d service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:27.005875 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ca10a-11c0-450c-a4c0-62103f11624d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:27.068307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ca10a-11c0-450c-a4c0-62103f11624d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:27.069820 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ca10a-11c0-450c-a4c0-62103f11624d service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:27.082302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1232/2481] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:27.728444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733aaee-5cf1-449d-b1c4-881529ec1407 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:27.737454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733aaee-5cf1-449d-b1c4-881529ec1407 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:27.741112 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae3638d-9f0b-4c76-b1c7-972d4b9eaafd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:27.741712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3638d-9f0b-4c76-b1c7-972d4b9eaafd service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:27.741997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3638d-9f0b-4c76-b1c7-972d4b9eaafd service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:27.742262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae3638d-9f0b-4c76-b1c7-972d4b9eaafd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:27.751110 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3638d-9f0b-4c76-b1c7-972d4b9eaafd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:27.752564 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae3638d-9f0b-4c76-b1c7-972d4b9eaafd service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:27.761889 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1250/2482] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:30.827459 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a7588-07f8-4c7c-aa97-fb96831ef565 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:30.828034 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a7588-07f8-4c7c-aa97-fb96831ef565 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:30.828034 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a7588-07f8-4c7c-aa97-fb96831ef565 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:30.829408 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1233/2483] 10.222.0.60 () {54 vars in 921 bytes} [Mon Sep 19 14:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:58:30.837205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae4499-1202-4701-8377-4d2d35f6a7ad None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:30.837724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae4499-1202-4701-8377-4d2d35f6a7ad None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:30.837821 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae4499-1202-4701-8377-4d2d35f6a7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:30.860535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a7ae4499-1202-4701-8377-4d2d35f6a7ad None None] MFA Rules not processed for user `59b5d2fd362b417c9b9be75f88a667ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:30.861968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a7ae4499-1202-4701-8377-4d2d35f6a7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:30.897206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:30.948281 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1251/2484] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Sep 19 14:58:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:30.960997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a7588-07f8-4c7c-aa97-fb96831ef565 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:30.963111 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d06a7588-07f8-4c7c-aa97-fb96831ef565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:31.026366 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a7588-07f8-4c7c-aa97-fb96831ef565 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:31.029972 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9703f5-ed8a-4242-9a9f-86ef9333f487 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caab7aaa72634b358d717768fbcb48d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:31.030707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9703f5-ed8a-4242-9a9f-86ef9333f487 service glance] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:31.031404 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9703f5-ed8a-4242-9a9f-86ef9333f487 service glance] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:31.031404 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9703f5-ed8a-4242-9a9f-86ef9333f487 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:31.042363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9703f5-ed8a-4242-9a9f-86ef9333f487 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:31.045107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9703f5-ed8a-4242-9a9f-86ef9333f487 service glance] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:31.055146 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1234/2485] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:58:31.132095 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462b82c-2d47-46f7-8ccd-fc26f9e0f12e None None] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:31.132478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462b82c-2d47-46f7-8ccd-fc26f9e0f12e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:31.132828 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462b82c-2d47-46f7-8ccd-fc26f9e0f12e None None] PATH_INFO: `/` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:31.134260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1252/2486] 10.222.0.60 () {54 vars in 921 bytes} [Mon Sep 19 14:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 19 14:58:31.141144 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedb523-e392-4da2-a9d5-639cd8f17d21 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:31.141490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedb523-e392-4da2-a9d5-639cd8f17d21 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:31.141779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedb523-e392-4da2-a9d5-639cd8f17d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:31.169144 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8eedb523-e392-4da2-a9d5-639cd8f17d21 None None] MFA Rules not processed for user `2fbef7b24f134f5eb1193aed1b7ee69c`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:31.170367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8eedb523-e392-4da2-a9d5-639cd8f17d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:31.227075 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1235/2487] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Sep 19 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:31.239572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462b82c-2d47-46f7-8ccd-fc26f9e0f12e None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:31.241303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e462b82c-2d47-46f7-8ccd-fc26f9e0f12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:31.255428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462b82c-2d47-46f7-8ccd-fc26f9e0f12e None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:31.259093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aacc8f-41d1-48f7-bdd3-6a275447cf2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:31.259700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aacc8f-41d1-48f7-bdd3-6a275447cf2e service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:31.260015 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aacc8f-41d1-48f7-bdd3-6a275447cf2e service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:31.260481 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aacc8f-41d1-48f7-bdd3-6a275447cf2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:31.268102 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aacc8f-41d1-48f7-bdd3-6a275447cf2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:31.269691 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aacc8f-41d1-48f7-bdd3-6a275447cf2e service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:31.277907 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1253/2488] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:31.363668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31514248-816e-47a8-a7d6-13a46794e865 None None] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:31.364012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31514248-816e-47a8-a7d6-13a46794e865 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:31.364289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31514248-816e-47a8-a7d6-13a46794e865 None None] PATH_INFO: `/` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:31.365655 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1236/2489] 10.222.0.60 () {54 vars in 921 bytes} [Mon Sep 19 14:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 19 14:58:31.370741 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548cde3-de09-4e62-9458-e3a2679699c3 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:31.371145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548cde3-de09-4e62-9458-e3a2679699c3 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:31.371498 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548cde3-de09-4e62-9458-e3a2679699c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:31.391439 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2548cde3-de09-4e62-9458-e3a2679699c3 None None] MFA Rules not processed for user `59b5d2fd362b417c9b9be75f88a667ad`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:31.393221 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2548cde3-de09-4e62-9458-e3a2679699c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:31.402858 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1254/2490] 10.222.0.60 () {58 vars in 1018 bytes} [Mon Sep 19 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:58:31.413938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31514248-816e-47a8-a7d6-13a46794e865 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:31.425814 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31514248-816e-47a8-a7d6-13a46794e865 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:31.429502 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915fc6b-a6d3-45a5-a64f-11a6422766aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:31.430258 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915fc6b-a6d3-45a5-a64f-11a6422766aa service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:31.430535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915fc6b-a6d3-45a5-a64f-11a6422766aa service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:31.430806 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915fc6b-a6d3-45a5-a64f-11a6422766aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:31.441723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915fc6b-a6d3-45a5-a64f-11a6422766aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:31.443165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915fc6b-a6d3-45a5-a64f-11a6422766aa service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:31.454303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1237/2491] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:31.742774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548cde3-de09-4e62-9458-e3a2679699c3 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:31.807439 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2548cde3-de09-4e62-9458-e3a2679699c3 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:31.812195 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a219c-a0d3-4c53-a172-f98b9bd29a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:31.812857 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a219c-a0d3-4c53-a172-f98b9bd29a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:31.813139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a219c-a0d3-4c53-a172-f98b9bd29a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:31.813413 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a219c-a0d3-4c53-a172-f98b9bd29a60 admin admin] PATH_INFO: `/v3/users/6062b03fcfad42b9bd07df0cd42e4f30` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:31.815061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a219c-a0d3-4c53-a172-f98b9bd29a60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6062b03fcfad42b9bd07df0cd42e4f30)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:31.817084 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a219c-a0d3-4c53-a172-f98b9bd29a60 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:31.867676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-256a219c-a0d3-4c53-a172-f98b9bd29a60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6062b03fcfad42b9bd07df0cd42e4f30'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:31.875118 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-256a219c-a0d3-4c53-a172-f98b9bd29a60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6062b03fcfad42b9bd07df0cd42e4f30'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:31.894528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-256a219c-a0d3-4c53-a172-f98b9bd29a60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6062b03fcfad42b9bd07df0cd42e4f30'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:31.905245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-256a219c-a0d3-4c53-a172-f98b9bd29a60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6062b03fcfad42b9bd07df0cd42e4f30'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:31.905895 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-256a219c-a0d3-4c53-a172-f98b9bd29a60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6062b03fcfad42b9bd07df0cd42e4f30'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:31.915409 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1255/2492] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:58:31 2022] DELETE /identity/v3/users/6062b03fcfad42b9bd07df0cd42e4f30 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:31.917372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f57f4-d722-417f-943c-4194fc324240 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:31.917849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f57f4-d722-417f-943c-4194fc324240 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:31.917969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f57f4-d722-417f-943c-4194fc324240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:31.926639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:31.940433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-671f57f4-d722-417f-943c-4194fc324240 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:31.941719 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-671f57f4-d722-417f-943c-4194fc324240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:32.006871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1238/2493] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:58:32.017216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a219c-a0d3-4c53-a172-f98b9bd29a60 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:32.019126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-256a219c-a0d3-4c53-a172-f98b9bd29a60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:32.045186 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:32.088380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a219c-a0d3-4c53-a172-f98b9bd29a60 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:32.092669 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcf8ff-7e02-4d0e-9999-7bf8e2ac0bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:32.093414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcf8ff-7e02-4d0e-9999-7bf8e2ac0bbb service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:32.093782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcf8ff-7e02-4d0e-9999-7bf8e2ac0bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:32.094155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effcf8ff-7e02-4d0e-9999-7bf8e2ac0bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:32.104665 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcf8ff-7e02-4d0e-9999-7bf8e2ac0bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:32.106135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effcf8ff-7e02-4d0e-9999-7bf8e2ac0bbb service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:32.135101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1256/2494] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:58:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:58:32.436815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f57f4-d722-417f-943c-4194fc324240 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:32.438353 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-671f57f4-d722-417f-943c-4194fc324240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:32.451493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f57f4-d722-417f-943c-4194fc324240 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:32.456135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644902c8-bf4e-4be7-a634-a35025b8e758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:32.456767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644902c8-bf4e-4be7-a634-a35025b8e758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:32.457045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644902c8-bf4e-4be7-a634-a35025b8e758 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:32.457312 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644902c8-bf4e-4be7-a634-a35025b8e758 admin admin] PATH_INFO: `/v3/projects/d40aaaba97d14586ade17642f9444b03` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:32.458415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644902c8-bf4e-4be7-a634-a35025b8e758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d40aaaba97d14586ade17642f9444b03)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:32.460363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644902c8-bf4e-4be7-a634-a35025b8e758 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:32.513245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-644902c8-bf4e-4be7-a634-a35025b8e758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd40aaaba97d14586ade17642f9444b03', 'request_id': 'req-644902c8-bf4e-4be7-a634-a35025b8e758'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:32.520704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-644902c8-bf4e-4be7-a634-a35025b8e758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd40aaaba97d14586ade17642f9444b03', 'request_id': 'req-644902c8-bf4e-4be7-a634-a35025b8e758'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:32.529045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-644902c8-bf4e-4be7-a634-a35025b8e758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd40aaaba97d14586ade17642f9444b03', 'request_id': 'req-644902c8-bf4e-4be7-a634-a35025b8e758'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:32.540850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-644902c8-bf4e-4be7-a634-a35025b8e758 admin admin] The token cache is being invalidate because project d40aaaba97d14586ade17642f9444b03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:58:32.541766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-644902c8-bf4e-4be7-a634-a35025b8e758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:32.542464 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-644902c8-bf4e-4be7-a634-a35025b8e758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:32.544189 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1239/2495] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:58:32 2022] DELETE /identity/v3/projects/d40aaaba97d14586ade17642f9444b03 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:32.548242 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcf8ff-7e02-4d0e-9999-7bf8e2ac0bbb service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:32.558726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effcf8ff-7e02-4d0e-9999-7bf8e2ac0bbb service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:32.622271 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a406b7-3ef0-441f-9cb9-6fb9ea5b5d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:32.623050 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a406b7-3ef0-441f-9cb9-6fb9ea5b5d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:32.623326 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a406b7-3ef0-441f-9cb9-6fb9ea5b5d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:32.623598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a406b7-3ef0-441f-9cb9-6fb9ea5b5d9d admin admin] PATH_INFO: `/v3/users/1ffcbce8487045458d9ebd34af9163a6` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:32.624756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a406b7-3ef0-441f-9cb9-6fb9ea5b5d9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ffcbce8487045458d9ebd34af9163a6)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:32.627268 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a406b7-3ef0-441f-9cb9-6fb9ea5b5d9d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:32.681657 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-04a406b7-3ef0-441f-9cb9-6fb9ea5b5d9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ffcbce8487045458d9ebd34af9163a6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:32.688752 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-04a406b7-3ef0-441f-9cb9-6fb9ea5b5d9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ffcbce8487045458d9ebd34af9163a6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:32.700912 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-04a406b7-3ef0-441f-9cb9-6fb9ea5b5d9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ffcbce8487045458d9ebd34af9163a6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:32.710541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-04a406b7-3ef0-441f-9cb9-6fb9ea5b5d9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ffcbce8487045458d9ebd34af9163a6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:32.711310 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-04a406b7-3ef0-441f-9cb9-6fb9ea5b5d9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ffcbce8487045458d9ebd34af9163a6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:32.718048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1257/2496] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:58:32 2022] DELETE /identity/v3/users/1ffcbce8487045458d9ebd34af9163a6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:33.003540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644902c8-bf4e-4be7-a634-a35025b8e758 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:33.006065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-644902c8-bf4e-4be7-a634-a35025b8e758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:33.031848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644902c8-bf4e-4be7-a634-a35025b8e758 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:33.104870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4908892-f430-4514-89da-d1af008b13d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:33.105520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4908892-f430-4514-89da-d1af008b13d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:33.105831 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4908892-f430-4514-89da-d1af008b13d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:33.106116 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4908892-f430-4514-89da-d1af008b13d5 admin admin] PATH_INFO: `/v3/projects/ecc5df3b30e74353b2064020a051b9ca` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:33.107414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4908892-f430-4514-89da-d1af008b13d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecc5df3b30e74353b2064020a051b9ca)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:33.109504 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4908892-f430-4514-89da-d1af008b13d5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:33.144522 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:33.160109 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a4908892-f430-4514-89da-d1af008b13d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecc5df3b30e74353b2064020a051b9ca', 'request_id': 'req-a4908892-f430-4514-89da-d1af008b13d5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:33.166729 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a4908892-f430-4514-89da-d1af008b13d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecc5df3b30e74353b2064020a051b9ca', 'request_id': 'req-a4908892-f430-4514-89da-d1af008b13d5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:33.175050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a4908892-f430-4514-89da-d1af008b13d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecc5df3b30e74353b2064020a051b9ca', 'request_id': 'req-a4908892-f430-4514-89da-d1af008b13d5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:33.186107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a4908892-f430-4514-89da-d1af008b13d5 admin admin] The token cache is being invalidate because project ecc5df3b30e74353b2064020a051b9ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:58:33.186419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a4908892-f430-4514-89da-d1af008b13d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:33.187098 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-a4908892-f430-4514-89da-d1af008b13d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:33.188636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1240/2497] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:58:33 2022] DELETE /identity/v3/projects/ecc5df3b30e74353b2064020a051b9ca => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:33.193325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a406b7-3ef0-441f-9cb9-6fb9ea5b5d9d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:33.199341 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-04a406b7-3ef0-441f-9cb9-6fb9ea5b5d9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:33.215987 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a406b7-3ef0-441f-9cb9-6fb9ea5b5d9d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:33.268277 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:33.290616 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de99a8-3c20-4240-b515-adc3fd65552b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:33.291263 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de99a8-3c20-4240-b515-adc3fd65552b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:33.291539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de99a8-3c20-4240-b515-adc3fd65552b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:33.291870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de99a8-3c20-4240-b515-adc3fd65552b admin admin] PATH_INFO: `/v3/users/8352c3a3800d40a5a221c6803194fe61` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:33.292962 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de99a8-3c20-4240-b515-adc3fd65552b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8352c3a3800d40a5a221c6803194fe61)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:33.295605 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de99a8-3c20-4240-b515-adc3fd65552b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:33.367672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c5de99a8-3c20-4240-b515-adc3fd65552b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8352c3a3800d40a5a221c6803194fe61'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:33.377519 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c5de99a8-3c20-4240-b515-adc3fd65552b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8352c3a3800d40a5a221c6803194fe61'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:33.391319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c5de99a8-3c20-4240-b515-adc3fd65552b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8352c3a3800d40a5a221c6803194fe61'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:33.401013 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c5de99a8-3c20-4240-b515-adc3fd65552b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8352c3a3800d40a5a221c6803194fe61'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:33.401901 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c5de99a8-3c20-4240-b515-adc3fd65552b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8352c3a3800d40a5a221c6803194fe61'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:33.414164 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1258/2498] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:58:33 2022] DELETE /identity/v3/users/8352c3a3800d40a5a221c6803194fe61 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:33.804660 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4908892-f430-4514-89da-d1af008b13d5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:33.806339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a4908892-f430-4514-89da-d1af008b13d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:33.824134 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4908892-f430-4514-89da-d1af008b13d5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:33.836527 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79de3c9-541b-4cee-85d3-7a99e08b5c5b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:33.836848 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79de3c9-541b-4cee-85d3-7a99e08b5c5b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:33.837762 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79de3c9-541b-4cee-85d3-7a99e08b5c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:33.869669 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c79de3c9-541b-4cee-85d3-7a99e08b5c5b None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:33.871601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c79de3c9-541b-4cee-85d3-7a99e08b5c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:33.910309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee538e-e51c-4b5f-983b-6bcb0d1c1cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:33.911151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee538e-e51c-4b5f-983b-6bcb0d1c1cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:33.911698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee538e-e51c-4b5f-983b-6bcb0d1c1cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:33.912095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee538e-e51c-4b5f-983b-6bcb0d1c1cd1 admin admin] PATH_INFO: `/v3/projects/70a84a2cb3bc455ba1a1d90cd5913a0e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:33.913916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee538e-e51c-4b5f-983b-6bcb0d1c1cd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70a84a2cb3bc455ba1a1d90cd5913a0e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:33.916275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee538e-e51c-4b5f-983b-6bcb0d1c1cd1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:33.954284 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1259/2499] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Sep 19 14:58:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:33.969718 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79de3c9-541b-4cee-85d3-7a99e08b5c5b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:33.971773 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c79de3c9-541b-4cee-85d3-7a99e08b5c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:33.976678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-54ee538e-e51c-4b5f-983b-6bcb0d1c1cd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70a84a2cb3bc455ba1a1d90cd5913a0e', 'request_id': 'req-54ee538e-e51c-4b5f-983b-6bcb0d1c1cd1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:33.984181 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-54ee538e-e51c-4b5f-983b-6bcb0d1c1cd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70a84a2cb3bc455ba1a1d90cd5913a0e', 'request_id': 'req-54ee538e-e51c-4b5f-983b-6bcb0d1c1cd1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:33.992235 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-54ee538e-e51c-4b5f-983b-6bcb0d1c1cd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70a84a2cb3bc455ba1a1d90cd5913a0e', 'request_id': 'req-54ee538e-e51c-4b5f-983b-6bcb0d1c1cd1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:34.004497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-54ee538e-e51c-4b5f-983b-6bcb0d1c1cd1 admin admin] The token cache is being invalidate because project 70a84a2cb3bc455ba1a1d90cd5913a0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:58:34.005107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-54ee538e-e51c-4b5f-983b-6bcb0d1c1cd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:34.005492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-54ee538e-e51c-4b5f-983b-6bcb0d1c1cd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:58:34.007044 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1241/2500] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:58:33 2022] DELETE /identity/v3/projects/70a84a2cb3bc455ba1a1d90cd5913a0e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:58:34.050298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79de3c9-541b-4cee-85d3-7a99e08b5c5b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:34.055692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:34.136063 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb134ea-3c38-44af-9d72-1b425d6ded76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:34.136691 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb134ea-3c38-44af-9d72-1b425d6ded76 service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:34.136976 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb134ea-3c38-44af-9d72-1b425d6ded76 service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:34.137252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb134ea-3c38-44af-9d72-1b425d6ded76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:34.139109 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4cb134ea-3c38-44af-9d72-1b425d6ded76 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:34.207179 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb134ea-3c38-44af-9d72-1b425d6ded76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:34.208843 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb134ea-3c38-44af-9d72-1b425d6ded76 service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:34.218054 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1260/2501] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 14:58:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 19 14:58:44.000094 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=64,keystone:DELETE=12,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:44.214325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=50 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:58:48.325166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7af197-6d66-49d2-83b4-f67199d7cd41 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:48.325698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7af197-6d66-49d2-83b4-f67199d7cd41 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:48.325698 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7af197-6d66-49d2-83b4-f67199d7cd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:48.346914 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ae7af197-6d66-49d2-83b4-f67199d7cd41 None None] MFA Rules not processed for user `52173f24f42a481380b047abe76bca57`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:48.348238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ae7af197-6d66-49d2-83b4-f67199d7cd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:48.403583 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1242/2502] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:58:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:58:51.598704 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965f008-0b5a-488c-bf56-de3244482d22 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:51.599685 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965f008-0b5a-488c-bf56-de3244482d22 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:51.600021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965f008-0b5a-488c-bf56-de3244482d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:51.621666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a965f008-0b5a-488c-bf56-de3244482d22 None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:58:51.622897 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a965f008-0b5a-488c-bf56-de3244482d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:51.630698 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1261/2503] 10.222.0.60 () {58 vars in 967 bytes} [Mon Sep 19 14:58:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:58:51.643516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7af197-6d66-49d2-83b4-f67199d7cd41 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:58:51.644973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ae7af197-6d66-49d2-83b4-f67199d7cd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:58:51.660852 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7af197-6d66-49d2-83b4-f67199d7cd41 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:58:51.666675 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09f84e-1c51-4d19-8abe-36a0abccdeda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:58:51.667309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09f84e-1c51-4d19-8abe-36a0abccdeda service swift] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:58:51.667589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09f84e-1c51-4d19-8abe-36a0abccdeda service swift] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:58:51.667891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09f84e-1c51-4d19-8abe-36a0abccdeda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:58:51.677335 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09f84e-1c51-4d19-8abe-36a0abccdeda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:58:51.678894 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09f84e-1c51-4d19-8abe-36a0abccdeda service swift] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:58:51.688031 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1243/2504] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 14:58:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 14:59:01.615342 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:01.684308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=31 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:03.365688 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965f008-0b5a-488c-bf56-de3244482d22 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:03.367132 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a965f008-0b5a-488c-bf56-de3244482d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:03.427979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965f008-0b5a-488c-bf56-de3244482d22 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:03.431796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70411d-9f74-4e6c-990a-91af0251470a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:03.432807 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70411d-9f74-4e6c-990a-91af0251470a service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:03.433092 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70411d-9f74-4e6c-990a-91af0251470a service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:03.433368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70411d-9f74-4e6c-990a-91af0251470a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:03.435356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bc70411d-9f74-4e6c-990a-91af0251470a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:03.503618 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70411d-9f74-4e6c-990a-91af0251470a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:03.505181 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70411d-9f74-4e6c-990a-91af0251470a service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:03.515349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1262/2505] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:06.697980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09f84e-1c51-4d19-8abe-36a0abccdeda service swift] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:06.700349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7b09f84e-1c51-4d19-8abe-36a0abccdeda service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:06.724989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09f84e-1c51-4d19-8abe-36a0abccdeda service swift] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:06.731762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb20acb-0a36-4fd4-8b15-9dad2d2075f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:06.732624 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb20acb-0a36-4fd4-8b15-9dad2d2075f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:06.732917 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb20acb-0a36-4fd4-8b15-9dad2d2075f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:06.733192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb20acb-0a36-4fd4-8b15-9dad2d2075f1 admin admin] PATH_INFO: `/v3/users/61743a63def7451d9b739f3b17afe8b8` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:06.734568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb20acb-0a36-4fd4-8b15-9dad2d2075f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61743a63def7451d9b739f3b17afe8b8)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:06.737566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb20acb-0a36-4fd4-8b15-9dad2d2075f1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:06.789973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1fb20acb-0a36-4fd4-8b15-9dad2d2075f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61743a63def7451d9b739f3b17afe8b8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:06.797137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1fb20acb-0a36-4fd4-8b15-9dad2d2075f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61743a63def7451d9b739f3b17afe8b8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:06.810624 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1fb20acb-0a36-4fd4-8b15-9dad2d2075f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61743a63def7451d9b739f3b17afe8b8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:06.820812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1fb20acb-0a36-4fd4-8b15-9dad2d2075f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61743a63def7451d9b739f3b17afe8b8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:06.821483 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-1fb20acb-0a36-4fd4-8b15-9dad2d2075f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61743a63def7451d9b739f3b17afe8b8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:06.834703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1244/2506] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:59:06 2022] DELETE /identity/v3/users/61743a63def7451d9b739f3b17afe8b8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:59:07.032743 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70411d-9f74-4e6c-990a-91af0251470a service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:07.043351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70411d-9f74-4e6c-990a-91af0251470a service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:07.128336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b11a1-f17b-43a7-aef5-524dc4bf27d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:07.129638 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b11a1-f17b-43a7-aef5-524dc4bf27d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:07.129924 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b11a1-f17b-43a7-aef5-524dc4bf27d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:07.130205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b11a1-f17b-43a7-aef5-524dc4bf27d6 admin admin] PATH_INFO: `/v3/projects/b89056beeba940128ec6ff72b0c55b34` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:07.131385 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b11a1-f17b-43a7-aef5-524dc4bf27d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b89056beeba940128ec6ff72b0c55b34)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:07.133398 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b11a1-f17b-43a7-aef5-524dc4bf27d6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:07.189371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d03b11a1-f17b-43a7-aef5-524dc4bf27d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b89056beeba940128ec6ff72b0c55b34', 'request_id': 'req-d03b11a1-f17b-43a7-aef5-524dc4bf27d6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:07.191613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:07.196558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d03b11a1-f17b-43a7-aef5-524dc4bf27d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b89056beeba940128ec6ff72b0c55b34', 'request_id': 'req-d03b11a1-f17b-43a7-aef5-524dc4bf27d6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:07.204513 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d03b11a1-f17b-43a7-aef5-524dc4bf27d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b89056beeba940128ec6ff72b0c55b34', 'request_id': 'req-d03b11a1-f17b-43a7-aef5-524dc4bf27d6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:07.220783 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d03b11a1-f17b-43a7-aef5-524dc4bf27d6 admin admin] The token cache is being invalidate because project b89056beeba940128ec6ff72b0c55b34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:59:07.221101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d03b11a1-f17b-43a7-aef5-524dc4bf27d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:07.221767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-d03b11a1-f17b-43a7-aef5-524dc4bf27d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:07.223566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1263/2507] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:59:07 2022] DELETE /identity/v3/projects/b89056beeba940128ec6ff72b0c55b34 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:07.229128 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154e57c-5d82-40da-af23-aa6ac4d65273 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:07.229128 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154e57c-5d82-40da-af23-aa6ac4d65273 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:07.229486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154e57c-5d82-40da-af23-aa6ac4d65273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:07.254777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-2154e57c-5d82-40da-af23-aa6ac4d65273 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:07.256029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2154e57c-5d82-40da-af23-aa6ac4d65273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:07.318377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1245/2508] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:07.326354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a007e7-3f7d-48dc-9f24-5eef893aa00a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:07.326753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a007e7-3f7d-48dc-9f24-5eef893aa00a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:07.327097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a007e7-3f7d-48dc-9f24-5eef893aa00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:07.350035 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-95a007e7-3f7d-48dc-9f24-5eef893aa00a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:07.351278 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-95a007e7-3f7d-48dc-9f24-5eef893aa00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:07.359678 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1264/2509] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:59:07.363967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154e57c-5d82-40da-af23-aa6ac4d65273 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:07.374392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154e57c-5d82-40da-af23-aa6ac4d65273 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:07.380152 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac735142-c627-4e4f-8905-ae1d61d6337e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:07.380797 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac735142-c627-4e4f-8905-ae1d61d6337e admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:07.381127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac735142-c627-4e4f-8905-ae1d61d6337e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:07.381400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac735142-c627-4e4f-8905-ae1d61d6337e admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:07.382049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac735142-c627-4e4f-8905-ae1d61d6337e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:59:07.382345 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac735142-c627-4e4f-8905-ae1d61d6337e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:07.384026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac735142-c627-4e4f-8905-ae1d61d6337e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:07.397436 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1246/2510] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:59:07.402086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a007e7-3f7d-48dc-9f24-5eef893aa00a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:07.412779 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a007e7-3f7d-48dc-9f24-5eef893aa00a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:07.418093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcc2a5-1075-4e45-935c-64e8bf2864f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:07.418783 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcc2a5-1075-4e45-935c-64e8bf2864f9 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:07.419092 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcc2a5-1075-4e45-935c-64e8bf2864f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:07.419696 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcc2a5-1075-4e45-935c-64e8bf2864f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:07.420562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dcc2a5-1075-4e45-935c-64e8bf2864f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:07.423058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dcc2a5-1075-4e45-935c-64e8bf2864f9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:07.456846 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1265/2511] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:59:07 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:07.457952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac735142-c627-4e4f-8905-ae1d61d6337e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:07.468865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac735142-c627-4e4f-8905-ae1d61d6337e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:07.530608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d6cc1-fb5e-4617-8700-ed2840cb6f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:07.531245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d6cc1-fb5e-4617-8700-ed2840cb6f85 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:07.531574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d6cc1-fb5e-4617-8700-ed2840cb6f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:07.532507 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d6cc1-fb5e-4617-8700-ed2840cb6f85 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:07.533120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d6cc1-fb5e-4617-8700-ed2840cb6f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:07.535052 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d6cc1-fb5e-4617-8700-ed2840cb6f85 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:07.543340 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:07.562657 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1247/2512] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:59:07 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:07.564344 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcc2a5-1075-4e45-935c-64e8bf2864f9 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:07.574685 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcc2a5-1075-4e45-935c-64e8bf2864f9 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:07.579999 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae859d8-b392-47fb-be5a-fe704a1f8202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:07.580602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae859d8-b392-47fb-be5a-fe704a1f8202 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:07.580878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae859d8-b392-47fb-be5a-fe704a1f8202 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:07.581148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae859d8-b392-47fb-be5a-fe704a1f8202 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:07.581774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae859d8-b392-47fb-be5a-fe704a1f8202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:07.583356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae859d8-b392-47fb-be5a-fe704a1f8202 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:07.593884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1266/2513] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:07.598542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d6cc1-fb5e-4617-8700-ed2840cb6f85 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:07.608463 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d6cc1-fb5e-4617-8700-ed2840cb6f85 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:07.613181 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3214a31-68d7-4247-9a41-df455d6bc403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:07.613926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3214a31-68d7-4247-9a41-df455d6bc403 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:07.614051 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3214a31-68d7-4247-9a41-df455d6bc403 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:07.614284 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3214a31-68d7-4247-9a41-df455d6bc403 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:07.614914 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3214a31-68d7-4247-9a41-df455d6bc403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:07.616555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3214a31-68d7-4247-9a41-df455d6bc403 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:07.629382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1248/2514] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:07.634563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae859d8-b392-47fb-be5a-fe704a1f8202 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:07.645101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae859d8-b392-47fb-be5a-fe704a1f8202 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:07.649868 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea265ea-2859-4b16-aa5b-d3410e8c68e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:07.650520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea265ea-2859-4b16-aa5b-d3410e8c68e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:07.650910 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea265ea-2859-4b16-aa5b-d3410e8c68e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:07.651187 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea265ea-2859-4b16-aa5b-d3410e8c68e0 admin admin] PATH_INFO: `/v3/projects/886ca546827a4efbb3812111508ea460/users/25f3eefbe0324a1fadf73453c655d85c/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:07.667384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea265ea-2859-4b16-aa5b-d3410e8c68e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=886ca546827a4efbb3812111508ea460, user_id=25f3eefbe0324a1fadf73453c655d85c, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:07.670067 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea265ea-2859-4b16-aa5b-d3410e8c68e0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:07.684735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1267/2515] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:07 2022] PUT /identity/v3/projects/886ca546827a4efbb3812111508ea460/users/25f3eefbe0324a1fadf73453c655d85c/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:59:07.695069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40e87a-7a15-4419-b306-7bf0b82808f2 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:07.695594 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40e87a-7a15-4419-b306-7bf0b82808f2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:07.695916 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40e87a-7a15-4419-b306-7bf0b82808f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:07.726015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ff40e87a-7a15-4419-b306-7bf0b82808f2 None None] MFA Rules not processed for user `25f3eefbe0324a1fadf73453c655d85c`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:07.727212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ff40e87a-7a15-4419-b306-7bf0b82808f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:07.795995 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1249/2516] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:07.808102 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea265ea-2859-4b16-aa5b-d3410e8c68e0 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:07.809851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9ea265ea-2859-4b16-aa5b-d3410e8c68e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:07.888250 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea265ea-2859-4b16-aa5b-d3410e8c68e0 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:07.894526 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dbff9f-80ec-427e-871e-b9b992e9a933 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:07.895141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbff9f-80ec-427e-871e-b9b992e9a933 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:07.895414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbff9f-80ec-427e-871e-b9b992e9a933 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:07.895685 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dbff9f-80ec-427e-871e-b9b992e9a933 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:07.904227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dbff9f-80ec-427e-871e-b9b992e9a933 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:07.905816 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dbff9f-80ec-427e-871e-b9b992e9a933 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:07.915765 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1268/2517] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:08.136561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40e87a-7a15-4419-b306-7bf0b82808f2 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:08.200056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40e87a-7a15-4419-b306-7bf0b82808f2 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:08.205706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c091b47-6711-4502-a8a4-ae1bb967ce40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:08.206342 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c091b47-6711-4502-a8a4-ae1bb967ce40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:08.206619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c091b47-6711-4502-a8a4-ae1bb967ce40 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:08.206891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c091b47-6711-4502-a8a4-ae1bb967ce40 admin admin] PATH_INFO: `/v3/users/25f3eefbe0324a1fadf73453c655d85c` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:08.208200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c091b47-6711-4502-a8a4-ae1bb967ce40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25f3eefbe0324a1fadf73453c655d85c)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:08.210118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c091b47-6711-4502-a8a4-ae1bb967ce40 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:08.261434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-4c091b47-6711-4502-a8a4-ae1bb967ce40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25f3eefbe0324a1fadf73453c655d85c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:08.269201 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-4c091b47-6711-4502-a8a4-ae1bb967ce40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25f3eefbe0324a1fadf73453c655d85c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:08.279337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:08.282186 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-4c091b47-6711-4502-a8a4-ae1bb967ce40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25f3eefbe0324a1fadf73453c655d85c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:08.290122 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-4c091b47-6711-4502-a8a4-ae1bb967ce40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25f3eefbe0324a1fadf73453c655d85c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:08.290967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-4c091b47-6711-4502-a8a4-ae1bb967ce40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25f3eefbe0324a1fadf73453c655d85c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:08.298711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1250/2518] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:59:08 2022] DELETE /identity/v3/users/25f3eefbe0324a1fadf73453c655d85c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:08.303983 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3249c36-32be-4982-b298-64a221f1e8f2 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:08.304496 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3249c36-32be-4982-b298-64a221f1e8f2 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:08.304585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3249c36-32be-4982-b298-64a221f1e8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:08.325117 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c3249c36-32be-4982-b298-64a221f1e8f2 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:08.326327 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c3249c36-32be-4982-b298-64a221f1e8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:08.398485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1269/2519] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:08.408566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c091b47-6711-4502-a8a4-ae1bb967ce40 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:08.410588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-4c091b47-6711-4502-a8a4-ae1bb967ce40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:08.484086 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c091b47-6711-4502-a8a4-ae1bb967ce40 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:08.489496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5022a5ad-7dd4-4c86-863e-0d603d2f5941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:08.490370 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022a5ad-7dd4-4c86-863e-0d603d2f5941 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:08.490650 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022a5ad-7dd4-4c86-863e-0d603d2f5941 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:08.491070 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5022a5ad-7dd4-4c86-863e-0d603d2f5941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:08.502138 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022a5ad-7dd4-4c86-863e-0d603d2f5941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:08.503573 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5022a5ad-7dd4-4c86-863e-0d603d2f5941 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:08.514219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1251/2520] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:08.799524 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3249c36-32be-4982-b298-64a221f1e8f2 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:08.801025 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c3249c36-32be-4982-b298-64a221f1e8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:08.810326 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:08.824518 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3249c36-32be-4982-b298-64a221f1e8f2 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:08.840098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5196ecb-874b-4f70-99d0-438e1764b6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:08.841515 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5196ecb-874b-4f70-99d0-438e1764b6ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:08.841897 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5196ecb-874b-4f70-99d0-438e1764b6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:08.842901 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5196ecb-874b-4f70-99d0-438e1764b6ef admin admin] PATH_INFO: `/v3/projects/886ca546827a4efbb3812111508ea460` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:08.843532 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5196ecb-874b-4f70-99d0-438e1764b6ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=886ca546827a4efbb3812111508ea460)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:08.845749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5196ecb-874b-4f70-99d0-438e1764b6ef admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:08.907425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a5196ecb-874b-4f70-99d0-438e1764b6ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '886ca546827a4efbb3812111508ea460', 'request_id': 'req-a5196ecb-874b-4f70-99d0-438e1764b6ef'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:08.913336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a5196ecb-874b-4f70-99d0-438e1764b6ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '886ca546827a4efbb3812111508ea460', 'request_id': 'req-a5196ecb-874b-4f70-99d0-438e1764b6ef'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:08.920399 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a5196ecb-874b-4f70-99d0-438e1764b6ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '886ca546827a4efbb3812111508ea460', 'request_id': 'req-a5196ecb-874b-4f70-99d0-438e1764b6ef'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:08.934495 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a5196ecb-874b-4f70-99d0-438e1764b6ef admin admin] The token cache is being invalidate because project 886ca546827a4efbb3812111508ea460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:59:08.935065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a5196ecb-874b-4f70-99d0-438e1764b6ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:08.935851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a5196ecb-874b-4f70-99d0-438e1764b6ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:08.939950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1270/2521] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:59:08 2022] DELETE /identity/v3/projects/886ca546827a4efbb3812111508ea460 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:59:08.942912 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22140106-3841-4e1b-9c66-43a4b4565960 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:08.943390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22140106-3841-4e1b-9c66-43a4b4565960 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:08.943619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22140106-3841-4e1b-9c66-43a4b4565960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:08.965629 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-22140106-3841-4e1b-9c66-43a4b4565960 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:08.966389 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-22140106-3841-4e1b-9c66-43a4b4565960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:09.040350 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1252/2522] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:09.052156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162bc61-275c-41a8-a912-660dee4b74e6 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:09.052435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162bc61-275c-41a8-a912-660dee4b74e6 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:09.052712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162bc61-275c-41a8-a912-660dee4b74e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:09.088998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9162bc61-275c-41a8-a912-660dee4b74e6 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:09.090952 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9162bc61-275c-41a8-a912-660dee4b74e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:09.108202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1271/2523] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:09.112505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22140106-3841-4e1b-9c66-43a4b4565960 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:09.122494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22140106-3841-4e1b-9c66-43a4b4565960 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:09.127280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876453e-4bf9-482c-9eae-33ac156ff2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:09.127931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876453e-4bf9-482c-9eae-33ac156ff2cb admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:09.128213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876453e-4bf9-482c-9eae-33ac156ff2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:09.128488 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876453e-4bf9-482c-9eae-33ac156ff2cb admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:09.129124 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876453e-4bf9-482c-9eae-33ac156ff2cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:59:09.129423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876453e-4bf9-482c-9eae-33ac156ff2cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:09.131024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876453e-4bf9-482c-9eae-33ac156ff2cb admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:09.144034 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1253/2524] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:59:09.148354 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162bc61-275c-41a8-a912-660dee4b74e6 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:09.158548 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162bc61-275c-41a8-a912-660dee4b74e6 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:09.164220 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90084a95-523a-4c1a-8040-49957fc6b5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:09.164851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90084a95-523a-4c1a-8040-49957fc6b5a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:09.165132 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90084a95-523a-4c1a-8040-49957fc6b5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:09.165405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90084a95-523a-4c1a-8040-49957fc6b5a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:09.166054 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90084a95-523a-4c1a-8040-49957fc6b5a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:09.168218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90084a95-523a-4c1a-8040-49957fc6b5a0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:09.201560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1272/2525] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:59:09 2022] POST /identity/v3/projects => generated 361 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:09.207191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876453e-4bf9-482c-9eae-33ac156ff2cb admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:09.218223 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876453e-4bf9-482c-9eae-33ac156ff2cb admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:09.276593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f346baf-5ce3-40f7-bf05-fdeebb16c4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:09.277208 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f346baf-5ce3-40f7-bf05-fdeebb16c4cc admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:09.277489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f346baf-5ce3-40f7-bf05-fdeebb16c4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:09.277767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f346baf-5ce3-40f7-bf05-fdeebb16c4cc admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:09.278390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f346baf-5ce3-40f7-bf05-fdeebb16c4cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:09.280342 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f346baf-5ce3-40f7-bf05-fdeebb16c4cc admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:09.302931 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1254/2526] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:59:09 2022] POST /identity/v3/users => generated 281 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:09.307436 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90084a95-523a-4c1a-8040-49957fc6b5a0 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:09.318350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90084a95-523a-4c1a-8040-49957fc6b5a0 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:09.323447 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f4da4-ac26-49c2-81db-c4a1ab011f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:09.324105 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f4da4-ac26-49c2-81db-c4a1ab011f56 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:09.324495 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f4da4-ac26-49c2-81db-c4a1ab011f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:09.324783 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f4da4-ac26-49c2-81db-c4a1ab011f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:09.325422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f4da4-ac26-49c2-81db-c4a1ab011f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:09.327022 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61f4da4-ac26-49c2-81db-c4a1ab011f56 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:09.338851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1273/2527] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:09.343590 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f346baf-5ce3-40f7-bf05-fdeebb16c4cc admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:09.354659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f346baf-5ce3-40f7-bf05-fdeebb16c4cc admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:09.360020 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21679c14-e764-447f-a152-994bed065992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:09.360899 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21679c14-e764-447f-a152-994bed065992 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:09.361183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21679c14-e764-447f-a152-994bed065992 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:09.361455 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21679c14-e764-447f-a152-994bed065992 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:09.363926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21679c14-e764-447f-a152-994bed065992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:09.365711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21679c14-e764-447f-a152-994bed065992 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:09.367578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:09.382001 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1255/2528] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:09.386828 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f4da4-ac26-49c2-81db-c4a1ab011f56 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:09.402025 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61f4da4-ac26-49c2-81db-c4a1ab011f56 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:09.413676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1bc45-a1a3-465a-8008-83f242fab212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:09.414505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1bc45-a1a3-465a-8008-83f242fab212 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:09.414790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1bc45-a1a3-465a-8008-83f242fab212 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:09.415068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d1bc45-a1a3-465a-8008-83f242fab212 admin admin] PATH_INFO: `/v3/projects/cbe8c7b384ce4f3d9ac665118f1a7c03/users/f224791936af4d49828459695f9c9289/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:09.431363 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1bc45-a1a3-465a-8008-83f242fab212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbe8c7b384ce4f3d9ac665118f1a7c03, user_id=f224791936af4d49828459695f9c9289, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:09.434331 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d1bc45-a1a3-465a-8008-83f242fab212 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:09.448171 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1274/2529] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:09 2022] PUT /identity/v3/projects/cbe8c7b384ce4f3d9ac665118f1a7c03/users/f224791936af4d49828459695f9c9289/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:59:09.459196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b455af-556a-4891-9c4b-c76d5a279a9b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:09.459485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b455af-556a-4891-9c4b-c76d5a279a9b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:09.459760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b455af-556a-4891-9c4b-c76d5a279a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:09.489125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f6b455af-556a-4891-9c4b-c76d5a279a9b None None] MFA Rules not processed for user `f224791936af4d49828459695f9c9289`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:09.490346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f6b455af-556a-4891-9c4b-c76d5a279a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:09.566095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1256/2530] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:09.570741 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1bc45-a1a3-465a-8008-83f242fab212 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:09.634915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d1bc45-a1a3-465a-8008-83f242fab212 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:09.639839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae70bd6-ec8d-4c8a-98a7-38624d4a2e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:09.640451 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae70bd6-ec8d-4c8a-98a7-38624d4a2e07 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:09.640729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae70bd6-ec8d-4c8a-98a7-38624d4a2e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:09.641083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae70bd6-ec8d-4c8a-98a7-38624d4a2e07 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:09.642440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae70bd6-ec8d-4c8a-98a7-38624d4a2e07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:09.644393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae70bd6-ec8d-4c8a-98a7-38624d4a2e07 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:09.671175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1275/2531] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:59:09 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:09.675691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b455af-556a-4891-9c4b-c76d5a279a9b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:09.686832 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b455af-556a-4891-9c4b-c76d5a279a9b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:09.750177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323a902-d0ff-483c-b195-112d723a192b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:09.751733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323a902-d0ff-483c-b195-112d723a192b admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:09.752093 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323a902-d0ff-483c-b195-112d723a192b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:09.752480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323a902-d0ff-483c-b195-112d723a192b admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:09.753443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323a902-d0ff-483c-b195-112d723a192b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:09.755878 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323a902-d0ff-483c-b195-112d723a192b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:09.780983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1257/2532] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:59:09 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:59:09.785101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae70bd6-ec8d-4c8a-98a7-38624d4a2e07 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:09.797651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae70bd6-ec8d-4c8a-98a7-38624d4a2e07 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:09.802259 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a2584-50aa-4128-938a-67a388411f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:09.802853 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a2584-50aa-4128-938a-67a388411f02 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:09.803131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a2584-50aa-4128-938a-67a388411f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:09.803402 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a2584-50aa-4128-938a-67a388411f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:09.804062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a2584-50aa-4128-938a-67a388411f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:09.805646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a2584-50aa-4128-938a-67a388411f02 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:09.806915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:09.820621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1276/2533] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:09.825721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323a902-d0ff-483c-b195-112d723a192b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:09.839343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323a902-d0ff-483c-b195-112d723a192b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:09.844912 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f350b9-4bb0-45f1-92e6-8d527ba020bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:09.845613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f350b9-4bb0-45f1-92e6-8d527ba020bd admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:09.845895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f350b9-4bb0-45f1-92e6-8d527ba020bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:09.846173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f350b9-4bb0-45f1-92e6-8d527ba020bd admin admin] PATH_INFO: `/v3/projects/e2c5afd305504b978cf25347b28f02ec/users/8367759063ac4917b174e7dc62e1d2c8/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:09.862115 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f350b9-4bb0-45f1-92e6-8d527ba020bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2c5afd305504b978cf25347b28f02ec, user_id=8367759063ac4917b174e7dc62e1d2c8, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:09.864914 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f350b9-4bb0-45f1-92e6-8d527ba020bd admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:09.878853 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1258/2534] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:09 2022] PUT /identity/v3/projects/e2c5afd305504b978cf25347b28f02ec/users/8367759063ac4917b174e7dc62e1d2c8/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:09.883134 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a2584-50aa-4128-938a-67a388411f02 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:09.893428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a2584-50aa-4128-938a-67a388411f02 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:09.954623 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3870c-7057-4afa-bc72-3c0ade1280ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:09.955235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3870c-7057-4afa-bc72-3c0ade1280ba admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:09.955520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3870c-7057-4afa-bc72-3c0ade1280ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:09.955796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3870c-7057-4afa-bc72-3c0ade1280ba admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:09.956463 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb3870c-7057-4afa-bc72-3c0ade1280ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:09.958053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb3870c-7057-4afa-bc72-3c0ade1280ba admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:09.968590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1277/2535] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:09 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:09.973342 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f350b9-4bb0-45f1-92e6-8d527ba020bd admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:09.984141 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f350b9-4bb0-45f1-92e6-8d527ba020bd admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:09.989437 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f1263-d269-486f-aa0e-30f7c2815b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:09.990092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f1263-d269-486f-aa0e-30f7c2815b09 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:09.990375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f1263-d269-486f-aa0e-30f7c2815b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:09.990651 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f1263-d269-486f-aa0e-30f7c2815b09 admin admin] PATH_INFO: `/v3/projects/e2c5afd305504b978cf25347b28f02ec/users/8367759063ac4917b174e7dc62e1d2c8/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:09.992630 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f1263-d269-486f-aa0e-30f7c2815b09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2c5afd305504b978cf25347b28f02ec, user_id=8367759063ac4917b174e7dc62e1d2c8, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:09.995327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f1263-d269-486f-aa0e-30f7c2815b09 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:10.006364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1259/2536] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:09 2022] PUT /identity/v3/projects/e2c5afd305504b978cf25347b28f02ec/users/8367759063ac4917b174e7dc62e1d2c8/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:10.016903 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ea125-2701-4af5-84b2-f27ddf25c3bb None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:10.017193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ea125-2701-4af5-84b2-f27ddf25c3bb None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:10.017474 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ea125-2701-4af5-84b2-f27ddf25c3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:10.045509 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-553ea125-2701-4af5-84b2-f27ddf25c3bb None None] MFA Rules not processed for user `8367759063ac4917b174e7dc62e1d2c8`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:10.046656 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-553ea125-2701-4af5-84b2-f27ddf25c3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:10.120367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1278/2537] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:10.134250 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f1263-d269-486f-aa0e-30f7c2815b09 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:10.135989 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a78f1263-d269-486f-aa0e-30f7c2815b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:10.180176 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:10.197499 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f1263-d269-486f-aa0e-30f7c2815b09 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:10.201197 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdbcb0-2736-4e76-a235-b340e5fd521b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:10.201801 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdbcb0-2736-4e76-a235-b340e5fd521b service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:10.202084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdbcb0-2736-4e76-a235-b340e5fd521b service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:10.202362 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bdbcb0-2736-4e76-a235-b340e5fd521b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:10.211647 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bdbcb0-2736-4e76-a235-b340e5fd521b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:10.213118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bdbcb0-2736-4e76-a235-b340e5fd521b service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:10.223493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1260/2538] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:10.299219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ea125-2701-4af5-84b2-f27ddf25c3bb None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:10.308964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ea125-2701-4af5-84b2-f27ddf25c3bb None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:10.312554 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94653f2b-fa74-458d-a257-fb439a79fa16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:10.313218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94653f2b-fa74-458d-a257-fb439a79fa16 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:10.313494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94653f2b-fa74-458d-a257-fb439a79fa16 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:10.313764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94653f2b-fa74-458d-a257-fb439a79fa16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:10.371804 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94653f2b-fa74-458d-a257-fb439a79fa16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:10.373264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94653f2b-fa74-458d-a257-fb439a79fa16 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:10.382171 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1279/2539] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:13.108735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdbcb0-2736-4e76-a235-b340e5fd521b service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:13.172133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bdbcb0-2736-4e76-a235-b340e5fd521b service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:13.177998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad48c9-00f9-45c6-8d56-f899a5ff7077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:13.178637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad48c9-00f9-45c6-8d56-f899a5ff7077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:13.178960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad48c9-00f9-45c6-8d56-f899a5ff7077 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:13.179191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad48c9-00f9-45c6-8d56-f899a5ff7077 admin admin] PATH_INFO: `/v3/users/f224791936af4d49828459695f9c9289` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:13.180473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad48c9-00f9-45c6-8d56-f899a5ff7077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f224791936af4d49828459695f9c9289)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:13.182666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad48c9-00f9-45c6-8d56-f899a5ff7077 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:13.234301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b6ad48c9-00f9-45c6-8d56-f899a5ff7077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f224791936af4d49828459695f9c9289'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:13.241922 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b6ad48c9-00f9-45c6-8d56-f899a5ff7077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f224791936af4d49828459695f9c9289'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:13.254843 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b6ad48c9-00f9-45c6-8d56-f899a5ff7077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f224791936af4d49828459695f9c9289'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:13.262548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b6ad48c9-00f9-45c6-8d56-f899a5ff7077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f224791936af4d49828459695f9c9289'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:13.263451 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-b6ad48c9-00f9-45c6-8d56-f899a5ff7077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f224791936af4d49828459695f9c9289'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:13.269779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1261/2540] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:59:13 2022] DELETE /identity/v3/users/f224791936af4d49828459695f9c9289 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:13.274386 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7bf79-86b2-4fa3-b07e-ec75e851df3e None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:13.274927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7bf79-86b2-4fa3-b07e-ec75e851df3e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:13.275031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7bf79-86b2-4fa3-b07e-ec75e851df3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:13.294868 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-55c7bf79-86b2-4fa3-b07e-ec75e851df3e None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:13.296133 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-55c7bf79-86b2-4fa3-b07e-ec75e851df3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:13.306595 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:13.367175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1280/2541] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:13.377549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad48c9-00f9-45c6-8d56-f899a5ff7077 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:13.379535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b6ad48c9-00f9-45c6-8d56-f899a5ff7077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:13.448990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad48c9-00f9-45c6-8d56-f899a5ff7077 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:13.455226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c73f8-2e44-43a9-b76d-149906080cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:13.456256 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c73f8-2e44-43a9-b76d-149906080cfe service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:13.456645 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c73f8-2e44-43a9-b76d-149906080cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:13.456974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c73f8-2e44-43a9-b76d-149906080cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:13.469884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c73f8-2e44-43a9-b76d-149906080cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:13.471345 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c73f8-2e44-43a9-b76d-149906080cfe service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:13.481317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1262/2542] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:13.760303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7bf79-86b2-4fa3-b07e-ec75e851df3e None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:13.761810 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-55c7bf79-86b2-4fa3-b07e-ec75e851df3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:13.773874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7bf79-86b2-4fa3-b07e-ec75e851df3e None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:13.778595 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a0dc2-82c7-413a-a343-f6362043044e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:13.779234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a0dc2-82c7-413a-a343-f6362043044e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:13.779510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a0dc2-82c7-413a-a343-f6362043044e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:13.779779 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a0dc2-82c7-413a-a343-f6362043044e admin admin] PATH_INFO: `/v3/projects/cbe8c7b384ce4f3d9ac665118f1a7c03` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:13.780919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a0dc2-82c7-413a-a343-f6362043044e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbe8c7b384ce4f3d9ac665118f1a7c03)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:13.782849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a0dc2-82c7-413a-a343-f6362043044e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:13.830529 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7c5a0dc2-82c7-413a-a343-f6362043044e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbe8c7b384ce4f3d9ac665118f1a7c03', 'request_id': 'req-7c5a0dc2-82c7-413a-a343-f6362043044e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:13.836048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7c5a0dc2-82c7-413a-a343-f6362043044e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbe8c7b384ce4f3d9ac665118f1a7c03', 'request_id': 'req-7c5a0dc2-82c7-413a-a343-f6362043044e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:13.842245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7c5a0dc2-82c7-413a-a343-f6362043044e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbe8c7b384ce4f3d9ac665118f1a7c03', 'request_id': 'req-7c5a0dc2-82c7-413a-a343-f6362043044e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:13.855397 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7c5a0dc2-82c7-413a-a343-f6362043044e admin admin] The token cache is being invalidate because project cbe8c7b384ce4f3d9ac665118f1a7c03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:59:13.855679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7c5a0dc2-82c7-413a-a343-f6362043044e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:13.856330 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7c5a0dc2-82c7-413a-a343-f6362043044e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:13.857794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1281/2543] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:59:13 2022] DELETE /identity/v3/projects/cbe8c7b384ce4f3d9ac665118f1a7c03 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:13.862554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c73f8-2e44-43a9-b76d-149906080cfe service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:13.867527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:13.874407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c73f8-2e44-43a9-b76d-149906080cfe service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:13.934739 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4ff9d-0140-478f-86aa-f8e517ffaa29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:13.935356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4ff9d-0140-478f-86aa-f8e517ffaa29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:13.935731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4ff9d-0140-478f-86aa-f8e517ffaa29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:13.935922 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4ff9d-0140-478f-86aa-f8e517ffaa29 admin admin] PATH_INFO: `/v3/users/8367759063ac4917b174e7dc62e1d2c8` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:13.937068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4ff9d-0140-478f-86aa-f8e517ffaa29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8367759063ac4917b174e7dc62e1d2c8)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:13.939607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4ff9d-0140-478f-86aa-f8e517ffaa29 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:13.983848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-19d4ff9d-0140-478f-86aa-f8e517ffaa29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8367759063ac4917b174e7dc62e1d2c8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:13.991782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-19d4ff9d-0140-478f-86aa-f8e517ffaa29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8367759063ac4917b174e7dc62e1d2c8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:14.003557 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-19d4ff9d-0140-478f-86aa-f8e517ffaa29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8367759063ac4917b174e7dc62e1d2c8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:14.010724 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-19d4ff9d-0140-478f-86aa-f8e517ffaa29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8367759063ac4917b174e7dc62e1d2c8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:14.011415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-19d4ff9d-0140-478f-86aa-f8e517ffaa29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8367759063ac4917b174e7dc62e1d2c8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:14.017390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1263/2544] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:59:13 2022] DELETE /identity/v3/users/8367759063ac4917b174e7dc62e1d2c8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:14.292340 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a0dc2-82c7-413a-a343-f6362043044e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:14.293803 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7c5a0dc2-82c7-413a-a343-f6362043044e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:14.307696 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a0dc2-82c7-413a-a343-f6362043044e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:14.376606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98490d-1d51-4fdf-b911-abebd0a4ec55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:14.377157 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98490d-1d51-4fdf-b911-abebd0a4ec55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:14.377436 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98490d-1d51-4fdf-b911-abebd0a4ec55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:14.377709 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98490d-1d51-4fdf-b911-abebd0a4ec55 admin admin] PATH_INFO: `/v3/projects/e2c5afd305504b978cf25347b28f02ec` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:14.378829 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98490d-1d51-4fdf-b911-abebd0a4ec55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2c5afd305504b978cf25347b28f02ec)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:14.381036 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98490d-1d51-4fdf-b911-abebd0a4ec55 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:14.426068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:14.433355 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-dd98490d-1d51-4fdf-b911-abebd0a4ec55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2c5afd305504b978cf25347b28f02ec', 'request_id': 'req-dd98490d-1d51-4fdf-b911-abebd0a4ec55'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:14.440253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-dd98490d-1d51-4fdf-b911-abebd0a4ec55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2c5afd305504b978cf25347b28f02ec', 'request_id': 'req-dd98490d-1d51-4fdf-b911-abebd0a4ec55'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:14.446466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-dd98490d-1d51-4fdf-b911-abebd0a4ec55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2c5afd305504b978cf25347b28f02ec', 'request_id': 'req-dd98490d-1d51-4fdf-b911-abebd0a4ec55'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:14.458472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-dd98490d-1d51-4fdf-b911-abebd0a4ec55 admin admin] The token cache is being invalidate because project e2c5afd305504b978cf25347b28f02ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:59:14.458759 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-dd98490d-1d51-4fdf-b911-abebd0a4ec55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:14.459374 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-dd98490d-1d51-4fdf-b911-abebd0a4ec55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:14.460994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1282/2545] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:59:14 2022] DELETE /identity/v3/projects/e2c5afd305504b978cf25347b28f02ec => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:14.466192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb768a-33ea-4731-b1dc-1a728cd5e67e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:14.466692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb768a-33ea-4731-b1dc-1a728cd5e67e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:14.466803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb768a-33ea-4731-b1dc-1a728cd5e67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:14.489723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1ebb768a-33ea-4731-b1dc-1a728cd5e67e None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:14.490955 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1ebb768a-33ea-4731-b1dc-1a728cd5e67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:14.550119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1264/2546] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:14.558347 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b1db8-7812-42cc-9f2a-2ee47848ec1c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:14.558633 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b1db8-7812-42cc-9f2a-2ee47848ec1c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:14.558908 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363b1db8-7812-42cc-9f2a-2ee47848ec1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:14.580024 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-363b1db8-7812-42cc-9f2a-2ee47848ec1c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:14.581230 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-363b1db8-7812-42cc-9f2a-2ee47848ec1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:14.589380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1283/2547] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:14.593559 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb768a-33ea-4731-b1dc-1a728cd5e67e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:14.603251 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb768a-33ea-4731-b1dc-1a728cd5e67e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:14.607782 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7c06c-b8d1-4f3a-84d6-a169eb336710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:14.608397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7c06c-b8d1-4f3a-84d6-a169eb336710 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:14.608672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7c06c-b8d1-4f3a-84d6-a169eb336710 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:14.608940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa7c06c-b8d1-4f3a-84d6-a169eb336710 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:14.609567 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7c06c-b8d1-4f3a-84d6-a169eb336710 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:59:14.610430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7c06c-b8d1-4f3a-84d6-a169eb336710 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:14.612063 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa7c06c-b8d1-4f3a-84d6-a169eb336710 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:14.628321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1265/2548] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:59:14.632734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b1db8-7812-42cc-9f2a-2ee47848ec1c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:14.642849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363b1db8-7812-42cc-9f2a-2ee47848ec1c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:14.648430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd047aa-47a5-4caa-b27e-5e6270668b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:14.649139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd047aa-47a5-4caa-b27e-5e6270668b62 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:14.649459 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd047aa-47a5-4caa-b27e-5e6270668b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:14.649777 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd047aa-47a5-4caa-b27e-5e6270668b62 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:14.650568 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd047aa-47a5-4caa-b27e-5e6270668b62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:14.653035 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd047aa-47a5-4caa-b27e-5e6270668b62 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:14.683629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1284/2549] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:59:14 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:14.687850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7c06c-b8d1-4f3a-84d6-a169eb336710 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:14.699049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa7c06c-b8d1-4f3a-84d6-a169eb336710 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:14.707632 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:14.766901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb463370-fa6d-4075-aabc-02abcd73e6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:14.766901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb463370-fa6d-4075-aabc-02abcd73e6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:14.767214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb463370-fa6d-4075-aabc-02abcd73e6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:14.767477 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb463370-fa6d-4075-aabc-02abcd73e6e5 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:14.768206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb463370-fa6d-4075-aabc-02abcd73e6e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:14.770157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb463370-fa6d-4075-aabc-02abcd73e6e5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:14.792173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1266/2550] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:59:14 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:14.796465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd047aa-47a5-4caa-b27e-5e6270668b62 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:14.807972 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd047aa-47a5-4caa-b27e-5e6270668b62 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:14.812667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439cdcf-9577-443c-b71b-9d38470c866b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:14.813268 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439cdcf-9577-443c-b71b-9d38470c866b admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:14.813554 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439cdcf-9577-443c-b71b-9d38470c866b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:14.813831 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439cdcf-9577-443c-b71b-9d38470c866b admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:14.814464 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439cdcf-9577-443c-b71b-9d38470c866b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:14.816088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b439cdcf-9577-443c-b71b-9d38470c866b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:14.825855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1285/2551] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:14.829759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb463370-fa6d-4075-aabc-02abcd73e6e5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:14.839608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb463370-fa6d-4075-aabc-02abcd73e6e5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:14.844980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c943cf4-b32d-451b-a889-c9ce80acd693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:14.845610 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c943cf4-b32d-451b-a889-c9ce80acd693 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:14.845892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c943cf4-b32d-451b-a889-c9ce80acd693 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:14.846166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c943cf4-b32d-451b-a889-c9ce80acd693 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:14.846804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c943cf4-b32d-451b-a889-c9ce80acd693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:14.848495 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c943cf4-b32d-451b-a889-c9ce80acd693 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:14.859140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1267/2552] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:14.864155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439cdcf-9577-443c-b71b-9d38470c866b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:14.874233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439cdcf-9577-443c-b71b-9d38470c866b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:14.879097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf29b8a-60ea-4971-9487-f2b86cbc0db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:14.879689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf29b8a-60ea-4971-9487-f2b86cbc0db1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:14.880054 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf29b8a-60ea-4971-9487-f2b86cbc0db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:14.880322 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf29b8a-60ea-4971-9487-f2b86cbc0db1 admin admin] PATH_INFO: `/v3/projects/038011378d0a47fdaa7510549976ae43/users/75ad9c2aa180424c95621c1b81a7b816/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:14.896223 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf29b8a-60ea-4971-9487-f2b86cbc0db1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=038011378d0a47fdaa7510549976ae43, user_id=75ad9c2aa180424c95621c1b81a7b816, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:14.898948 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf29b8a-60ea-4971-9487-f2b86cbc0db1 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:14.912324 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1286/2553] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:14 2022] PUT /identity/v3/projects/038011378d0a47fdaa7510549976ae43/users/75ad9c2aa180424c95621c1b81a7b816/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:14.923436 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2702d-8864-42b8-875f-aaa4771779d8 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:14.923726 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2702d-8864-42b8-875f-aaa4771779d8 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:14.924033 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2702d-8864-42b8-875f-aaa4771779d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:14.952418 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-79a2702d-8864-42b8-875f-aaa4771779d8 None None] MFA Rules not processed for user `75ad9c2aa180424c95621c1b81a7b816`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:14.953672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-79a2702d-8864-42b8-875f-aaa4771779d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:15.018466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1268/2554] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:15.023108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf29b8a-60ea-4971-9487-f2b86cbc0db1 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:15.103537 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf29b8a-60ea-4971-9487-f2b86cbc0db1 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:15.108703 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1bba4f-c3c5-4fd4-b699-5b964a830c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:15.109318 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bba4f-c3c5-4fd4-b699-5b964a830c18 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:15.109613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bba4f-c3c5-4fd4-b699-5b964a830c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:15.109924 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1bba4f-c3c5-4fd4-b699-5b964a830c18 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:15.110560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1bba4f-c3c5-4fd4-b699-5b964a830c18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:15.112503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1bba4f-c3c5-4fd4-b699-5b964a830c18 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:15.138756 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1287/2555] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:59:15 2022] POST /identity/v3/projects => generated 369 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:15.143335 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2702d-8864-42b8-875f-aaa4771779d8 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:15.153909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2702d-8864-42b8-875f-aaa4771779d8 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:15.210205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e86eb-8925-4bc9-b241-4aab235a666d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:15.210857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e86eb-8925-4bc9-b241-4aab235a666d admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:15.211096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e86eb-8925-4bc9-b241-4aab235a666d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:15.211373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e86eb-8925-4bc9-b241-4aab235a666d admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:15.212033 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e86eb-8925-4bc9-b241-4aab235a666d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:15.213954 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e86eb-8925-4bc9-b241-4aab235a666d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:15.235841 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1269/2556] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:59:15 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:15.239862 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1bba4f-c3c5-4fd4-b699-5b964a830c18 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:15.250667 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1bba4f-c3c5-4fd4-b699-5b964a830c18 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:15.255334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495a888-7f77-4398-8e84-ccd956cd84d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:15.255979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495a888-7f77-4398-8e84-ccd956cd84d8 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:15.256264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495a888-7f77-4398-8e84-ccd956cd84d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:15.256539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2495a888-7f77-4398-8e84-ccd956cd84d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:15.257171 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495a888-7f77-4398-8e84-ccd956cd84d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:15.258763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2495a888-7f77-4398-8e84-ccd956cd84d8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:15.269806 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1288/2557] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:15.273944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e86eb-8925-4bc9-b241-4aab235a666d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:15.284693 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e86eb-8925-4bc9-b241-4aab235a666d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:15.289311 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe59eb5-5754-45c7-8a25-aefaf50ea033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:15.289955 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe59eb5-5754-45c7-8a25-aefaf50ea033 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:15.290233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe59eb5-5754-45c7-8a25-aefaf50ea033 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:15.290508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe59eb5-5754-45c7-8a25-aefaf50ea033 admin admin] PATH_INFO: `/v3/projects/986a426925874b05851dca5ab5feb5af/users/2dd51c581d2a4d5aa1d68c2732998698/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:15.292729 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:15.306636 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe59eb5-5754-45c7-8a25-aefaf50ea033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=986a426925874b05851dca5ab5feb5af, user_id=2dd51c581d2a4d5aa1d68c2732998698, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:15.309596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe59eb5-5754-45c7-8a25-aefaf50ea033 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:15.322205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1270/2558] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:15 2022] PUT /identity/v3/projects/986a426925874b05851dca5ab5feb5af/users/2dd51c581d2a4d5aa1d68c2732998698/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:15.326151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495a888-7f77-4398-8e84-ccd956cd84d8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:15.336781 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2495a888-7f77-4398-8e84-ccd956cd84d8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:15.380638 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:15.402397 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0214670-8436-4ec0-9603-77bcdee8b81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:15.403015 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0214670-8436-4ec0-9603-77bcdee8b81c admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:15.403305 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0214670-8436-4ec0-9603-77bcdee8b81c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:15.403605 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0214670-8436-4ec0-9603-77bcdee8b81c admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:15.404537 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0214670-8436-4ec0-9603-77bcdee8b81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:15.406467 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0214670-8436-4ec0-9603-77bcdee8b81c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:15.418175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1289/2559] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:15.423017 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe59eb5-5754-45c7-8a25-aefaf50ea033 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:15.434137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe59eb5-5754-45c7-8a25-aefaf50ea033 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:15.438927 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c1394-2598-4eb0-a67e-47bd1d1cce1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:15.439589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c1394-2598-4eb0-a67e-47bd1d1cce1e admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:15.439589 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c1394-2598-4eb0-a67e-47bd1d1cce1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:15.440642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c1394-2598-4eb0-a67e-47bd1d1cce1e admin admin] PATH_INFO: `/v3/projects/986a426925874b05851dca5ab5feb5af/users/2dd51c581d2a4d5aa1d68c2732998698/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:15.442031 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c1394-2598-4eb0-a67e-47bd1d1cce1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=986a426925874b05851dca5ab5feb5af, user_id=2dd51c581d2a4d5aa1d68c2732998698, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:15.444773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5c1394-2598-4eb0-a67e-47bd1d1cce1e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:15.455982 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1271/2560] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:15 2022] PUT /identity/v3/projects/986a426925874b05851dca5ab5feb5af/users/2dd51c581d2a4d5aa1d68c2732998698/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:15.467281 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f43b9-f28d-447b-a70d-b7db067b5df0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:15.467567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f43b9-f28d-447b-a70d-b7db067b5df0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:15.467877 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f43b9-f28d-447b-a70d-b7db067b5df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:15.496504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-649f43b9-f28d-447b-a70d-b7db067b5df0 None None] MFA Rules not processed for user `2dd51c581d2a4d5aa1d68c2732998698`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:15.497736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-649f43b9-f28d-447b-a70d-b7db067b5df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:15.565622 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1290/2561] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:15.576775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c1394-2598-4eb0-a67e-47bd1d1cce1e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:15.579247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1d5c1394-2598-4eb0-a67e-47bd1d1cce1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:15.636833 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5c1394-2598-4eb0-a67e-47bd1d1cce1e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:15.640583 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263b1f9-33fa-4d88-8e24-fef060dac3a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:15.641775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263b1f9-33fa-4d88-8e24-fef060dac3a1 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:15.642057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263b1f9-33fa-4d88-8e24-fef060dac3a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:15.642339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263b1f9-33fa-4d88-8e24-fef060dac3a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:15.651481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263b1f9-33fa-4d88-8e24-fef060dac3a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:15.653003 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263b1f9-33fa-4d88-8e24-fef060dac3a1 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:15.662390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1272/2562] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:15.707516 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f43b9-f28d-447b-a70d-b7db067b5df0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:15.708960 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-649f43b9-f28d-447b-a70d-b7db067b5df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:15.721088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649f43b9-f28d-447b-a70d-b7db067b5df0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:15.725447 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483d4b9-5465-4bb3-b872-c8155e64fbf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:15.725792 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483d4b9-5465-4bb3-b872-c8155e64fbf8 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:15.726073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483d4b9-5465-4bb3-b872-c8155e64fbf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:15.726340 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483d4b9-5465-4bb3-b872-c8155e64fbf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:15.786563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483d4b9-5465-4bb3-b872-c8155e64fbf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:15.788037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483d4b9-5465-4bb3-b872-c8155e64fbf8 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:15.797766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1291/2563] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:19.131736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263b1f9-33fa-4d88-8e24-fef060dac3a1 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:19.194600 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263b1f9-33fa-4d88-8e24-fef060dac3a1 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:19.199155 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2405a22-230a-4fec-821b-9f67dc93b8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:19.199785 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2405a22-230a-4fec-821b-9f67dc93b8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:19.200106 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2405a22-230a-4fec-821b-9f67dc93b8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:19.200383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2405a22-230a-4fec-821b-9f67dc93b8e1 admin admin] PATH_INFO: `/v3/users/75ad9c2aa180424c95621c1b81a7b816` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:19.201499 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2405a22-230a-4fec-821b-9f67dc93b8e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75ad9c2aa180424c95621c1b81a7b816)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:19.203419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2405a22-230a-4fec-821b-9f67dc93b8e1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:19.251005 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c2405a22-230a-4fec-821b-9f67dc93b8e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75ad9c2aa180424c95621c1b81a7b816'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:19.257908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c2405a22-230a-4fec-821b-9f67dc93b8e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75ad9c2aa180424c95621c1b81a7b816'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:19.269779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c2405a22-230a-4fec-821b-9f67dc93b8e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75ad9c2aa180424c95621c1b81a7b816'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:19.271433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:19.278703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c2405a22-230a-4fec-821b-9f67dc93b8e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75ad9c2aa180424c95621c1b81a7b816'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:19.279954 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c2405a22-230a-4fec-821b-9f67dc93b8e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75ad9c2aa180424c95621c1b81a7b816'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:19.286762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1273/2564] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:59:19 2022] DELETE /identity/v3/users/75ad9c2aa180424c95621c1b81a7b816 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:19.291449 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc59fb1-9be4-4575-91fc-8994061145cb None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:19.291973 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc59fb1-9be4-4575-91fc-8994061145cb None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:19.292073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc59fb1-9be4-4575-91fc-8994061145cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:19.311752 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-bbc59fb1-9be4-4575-91fc-8994061145cb None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:19.313055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bbc59fb1-9be4-4575-91fc-8994061145cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:19.364537 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:19.373535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1292/2565] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:19.383379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2405a22-230a-4fec-821b-9f67dc93b8e1 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:19.384812 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c2405a22-230a-4fec-821b-9f67dc93b8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:19.447141 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2405a22-230a-4fec-821b-9f67dc93b8e1 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:19.450821 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8c034-c42c-4e16-916d-437c1d567485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:19.451430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8c034-c42c-4e16-916d-437c1d567485 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:19.451711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8c034-c42c-4e16-916d-437c1d567485 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:19.452021 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8c034-c42c-4e16-916d-437c1d567485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:19.461140 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8c034-c42c-4e16-916d-437c1d567485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:19.462575 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8c034-c42c-4e16-916d-437c1d567485 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:19.472599 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1274/2566] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:19.739814 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc59fb1-9be4-4575-91fc-8994061145cb None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:19.741312 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bbc59fb1-9be4-4575-91fc-8994061145cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:19.754450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc59fb1-9be4-4575-91fc-8994061145cb None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:19.759335 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:19.759877 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:19.760162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:19.760438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766 admin admin] PATH_INFO: `/v3/projects/038011378d0a47fdaa7510549976ae43` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:19.761641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=038011378d0a47fdaa7510549976ae43)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:19.763546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:19.810920 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '038011378d0a47fdaa7510549976ae43', 'request_id': 'req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:19.817300 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '038011378d0a47fdaa7510549976ae43', 'request_id': 'req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:19.823485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '038011378d0a47fdaa7510549976ae43', 'request_id': 'req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:19.835915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766 admin admin] The token cache is being invalidate because project 038011378d0a47fdaa7510549976ae43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:59:19.836206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:19.836830 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:19.838373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1293/2567] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:59:19 2022] DELETE /identity/v3/projects/038011378d0a47fdaa7510549976ae43 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:19.842564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8c034-c42c-4e16-916d-437c1d567485 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:19.852142 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8c034-c42c-4e16-916d-437c1d567485 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:19.908759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed599fb-a66d-4bbf-807c-a93e27fdec93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:19.909387 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed599fb-a66d-4bbf-807c-a93e27fdec93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:19.909658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed599fb-a66d-4bbf-807c-a93e27fdec93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:19.909924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed599fb-a66d-4bbf-807c-a93e27fdec93 admin admin] PATH_INFO: `/v3/users/2dd51c581d2a4d5aa1d68c2732998698` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:19.911029 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed599fb-a66d-4bbf-807c-a93e27fdec93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dd51c581d2a4d5aa1d68c2732998698)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:19.912972 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed599fb-a66d-4bbf-807c-a93e27fdec93 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:19.959777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bed599fb-a66d-4bbf-807c-a93e27fdec93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dd51c581d2a4d5aa1d68c2732998698'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:19.966805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bed599fb-a66d-4bbf-807c-a93e27fdec93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dd51c581d2a4d5aa1d68c2732998698'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:19.979513 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bed599fb-a66d-4bbf-807c-a93e27fdec93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dd51c581d2a4d5aa1d68c2732998698'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:19.987163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bed599fb-a66d-4bbf-807c-a93e27fdec93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dd51c581d2a4d5aa1d68c2732998698'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:19.987941 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bed599fb-a66d-4bbf-807c-a93e27fdec93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dd51c581d2a4d5aa1d68c2732998698'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:19.994190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1275/2568] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:59:19 2022] DELETE /identity/v3/users/2dd51c581d2a4d5aa1d68c2732998698 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:20.270730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:20.272348 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:20.286217 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9a7a2-82cd-476d-b6ec-2ace92c62766 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:20.343993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a8f42-1951-40e6-832a-63218cb67cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:20.344678 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a8f42-1951-40e6-832a-63218cb67cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:20.344920 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a8f42-1951-40e6-832a-63218cb67cec admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:20.345196 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a8f42-1951-40e6-832a-63218cb67cec admin admin] PATH_INFO: `/v3/projects/986a426925874b05851dca5ab5feb5af` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:20.346351 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a8f42-1951-40e6-832a-63218cb67cec admin admin] RBAC: Authorizing `identity:delete_project(project_id=986a426925874b05851dca5ab5feb5af)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:20.348301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a8f42-1951-40e6-832a-63218cb67cec admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:20.395165 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2a9a8f42-1951-40e6-832a-63218cb67cec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '986a426925874b05851dca5ab5feb5af', 'request_id': 'req-2a9a8f42-1951-40e6-832a-63218cb67cec'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:20.400894 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2a9a8f42-1951-40e6-832a-63218cb67cec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '986a426925874b05851dca5ab5feb5af', 'request_id': 'req-2a9a8f42-1951-40e6-832a-63218cb67cec'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:20.407012 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2a9a8f42-1951-40e6-832a-63218cb67cec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '986a426925874b05851dca5ab5feb5af', 'request_id': 'req-2a9a8f42-1951-40e6-832a-63218cb67cec'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:20.419373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2a9a8f42-1951-40e6-832a-63218cb67cec admin admin] The token cache is being invalidate because project 986a426925874b05851dca5ab5feb5af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:59:20.419582 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2a9a8f42-1951-40e6-832a-63218cb67cec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:20.420359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2a9a8f42-1951-40e6-832a-63218cb67cec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:20.421826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1294/2569] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:59:20 2022] DELETE /identity/v3/projects/986a426925874b05851dca5ab5feb5af => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:20.427292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bd033-461a-44fc-ada1-900b69880d60 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:20.427785 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bd033-461a-44fc-ada1-900b69880d60 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:20.427942 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157bd033-461a-44fc-ada1-900b69880d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:20.448419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-157bd033-461a-44fc-ada1-900b69880d60 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:20.449668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-157bd033-461a-44fc-ada1-900b69880d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:20.458274 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:20.511326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1276/2570] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:59:20.519188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f293a-23f3-4382-a5dc-fdf58495b896 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:20.519477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f293a-23f3-4382-a5dc-fdf58495b896 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:20.519799 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f293a-23f3-4382-a5dc-fdf58495b896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:20.539265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3f0f293a-23f3-4382-a5dc-fdf58495b896 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:20.540528 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3f0f293a-23f3-4382-a5dc-fdf58495b896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:20.548982 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1295/2571] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:20.553230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157bd033-461a-44fc-ada1-900b69880d60 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:20.563641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157bd033-461a-44fc-ada1-900b69880d60 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:20.568248 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02022c-036f-42f4-a935-405a919ad5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:20.568835 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02022c-036f-42f4-a935-405a919ad5c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:20.569114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02022c-036f-42f4-a935-405a919ad5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:20.569379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02022c-036f-42f4-a935-405a919ad5c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:20.570002 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02022c-036f-42f4-a935-405a919ad5c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:59:20.570299 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02022c-036f-42f4-a935-405a919ad5c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:20.571895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02022c-036f-42f4-a935-405a919ad5c7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:20.583224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1277/2572] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:59:20.587484 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f293a-23f3-4382-a5dc-fdf58495b896 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:20.596809 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f293a-23f3-4382-a5dc-fdf58495b896 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:20.601576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33ccbe-0eba-4645-8e61-aad4b4e7a96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:20.602170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33ccbe-0eba-4645-8e61-aad4b4e7a96b admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:20.602448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33ccbe-0eba-4645-8e61-aad4b4e7a96b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:20.602724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33ccbe-0eba-4645-8e61-aad4b4e7a96b admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:20.603346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33ccbe-0eba-4645-8e61-aad4b4e7a96b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:20.605263 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33ccbe-0eba-4645-8e61-aad4b4e7a96b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:20.610265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:20.636702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1296/2573] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:59:20 2022] POST /identity/v3/projects => generated 409 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:59:20.641038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02022c-036f-42f4-a935-405a919ad5c7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:20.651540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02022c-036f-42f4-a935-405a919ad5c7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:20.707336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc34d6-4ab4-4e6c-ab75-c8a027221c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:20.707871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc34d6-4ab4-4e6c-ab75-c8a027221c14 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:20.708154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc34d6-4ab4-4e6c-ab75-c8a027221c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:20.708424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc34d6-4ab4-4e6c-ab75-c8a027221c14 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:20.709049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc34d6-4ab4-4e6c-ab75-c8a027221c14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:20.711064 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc34d6-4ab4-4e6c-ab75-c8a027221c14 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:20.734091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1278/2574] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:59:20 2022] POST /identity/v3/users => generated 305 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:20.739154 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33ccbe-0eba-4645-8e61-aad4b4e7a96b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:20.749725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33ccbe-0eba-4645-8e61-aad4b4e7a96b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:20.754454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06425681-68a4-42c7-8071-07e730b5053d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:20.755052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06425681-68a4-42c7-8071-07e730b5053d admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:20.755326 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06425681-68a4-42c7-8071-07e730b5053d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:20.755597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06425681-68a4-42c7-8071-07e730b5053d admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:20.756303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06425681-68a4-42c7-8071-07e730b5053d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:20.757891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06425681-68a4-42c7-8071-07e730b5053d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:20.767713 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1297/2575] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:20.771685 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc34d6-4ab4-4e6c-ab75-c8a027221c14 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:20.782026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc34d6-4ab4-4e6c-ab75-c8a027221c14 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:20.786568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ef36a-dcb8-4d66-b098-f0aac0b285d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:20.787156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ef36a-dcb8-4d66-b098-f0aac0b285d7 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:20.787433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ef36a-dcb8-4d66-b098-f0aac0b285d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:20.787703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ef36a-dcb8-4d66-b098-f0aac0b285d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:20.788357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ef36a-dcb8-4d66-b098-f0aac0b285d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:20.789998 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ef36a-dcb8-4d66-b098-f0aac0b285d7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:20.801322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1279/2576] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:59:20.805448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06425681-68a4-42c7-8071-07e730b5053d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:20.815626 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06425681-68a4-42c7-8071-07e730b5053d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:20.820235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00255fde-1657-4b7f-bd14-c0f24e8ee15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:20.820879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00255fde-1657-4b7f-bd14-c0f24e8ee15b admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:20.821171 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00255fde-1657-4b7f-bd14-c0f24e8ee15b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:20.821426 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00255fde-1657-4b7f-bd14-c0f24e8ee15b admin admin] PATH_INFO: `/v3/projects/6aa16f68e0fd44449459c2d935e0cf0c/users/4854c2f5c20a494d9c5ef9bb46d8c42e/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:20.836460 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00255fde-1657-4b7f-bd14-c0f24e8ee15b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aa16f68e0fd44449459c2d935e0cf0c, user_id=4854c2f5c20a494d9c5ef9bb46d8c42e, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:20.839103 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00255fde-1657-4b7f-bd14-c0f24e8ee15b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:20.852822 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1298/2577] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:20 2022] PUT /identity/v3/projects/6aa16f68e0fd44449459c2d935e0cf0c/users/4854c2f5c20a494d9c5ef9bb46d8c42e/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:20.863664 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae8268-5e42-4421-ac6c-b2bf324edd81 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:20.863975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae8268-5e42-4421-ac6c-b2bf324edd81 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:20.864295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae8268-5e42-4421-ac6c-b2bf324edd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:20.893227 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-73ae8268-5e42-4421-ac6c-b2bf324edd81 None None] MFA Rules not processed for user `4854c2f5c20a494d9c5ef9bb46d8c42e`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:20.894454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-73ae8268-5e42-4421-ac6c-b2bf324edd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:20.960666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1280/2578] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:20 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:20.965394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00255fde-1657-4b7f-bd14-c0f24e8ee15b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:21.026192 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00255fde-1657-4b7f-bd14-c0f24e8ee15b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:21.030832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d3a7a-d678-4228-8998-e33eed6470d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:21.032021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d3a7a-d678-4228-8998-e33eed6470d3 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:21.032299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d3a7a-d678-4228-8998-e33eed6470d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:21.032569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d3a7a-d678-4228-8998-e33eed6470d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:21.033187 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d3a7a-d678-4228-8998-e33eed6470d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:21.035053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d3a7a-d678-4228-8998-e33eed6470d3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:21.063182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1299/2579] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:59:20 2022] POST /identity/v3/projects => generated 407 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:59:21.067596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae8268-5e42-4421-ac6c-b2bf324edd81 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:21.078857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae8268-5e42-4421-ac6c-b2bf324edd81 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:21.083861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:21.145188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc48028-4696-4538-8d0d-26a3acb8df37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:21.145794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc48028-4696-4538-8d0d-26a3acb8df37 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:21.146074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc48028-4696-4538-8d0d-26a3acb8df37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:21.146346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc48028-4696-4538-8d0d-26a3acb8df37 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:21.146978 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc48028-4696-4538-8d0d-26a3acb8df37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:21.148969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc48028-4696-4538-8d0d-26a3acb8df37 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:21.170458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1281/2580] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:59:21 2022] POST /identity/v3/users => generated 310 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:59:21.174579 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d3a7a-d678-4228-8998-e33eed6470d3 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:21.185386 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d3a7a-d678-4228-8998-e33eed6470d3 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:21.190416 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4703793-c5d6-4f60-95e2-90392e0f3a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:21.191021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4703793-c5d6-4f60-95e2-90392e0f3a79 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:21.191302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4703793-c5d6-4f60-95e2-90392e0f3a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:21.191577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4703793-c5d6-4f60-95e2-90392e0f3a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:21.192244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4703793-c5d6-4f60-95e2-90392e0f3a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:21.193832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4703793-c5d6-4f60-95e2-90392e0f3a79 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:21.204758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1300/2581] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:21.208985 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc48028-4696-4538-8d0d-26a3acb8df37 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:21.220042 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc48028-4696-4538-8d0d-26a3acb8df37 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:21.224712 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e223a-ad43-4def-9d38-c9feefda9e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:21.225391 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e223a-ad43-4def-9d38-c9feefda9e89 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:21.225674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e223a-ad43-4def-9d38-c9feefda9e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:21.225976 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e223a-ad43-4def-9d38-c9feefda9e89 admin admin] PATH_INFO: `/v3/projects/436befff7ac84a749cb0945ab6d3bc1b/users/8cf16c186db04fd7892a4d1381376599/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:21.241418 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e223a-ad43-4def-9d38-c9feefda9e89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=436befff7ac84a749cb0945ab6d3bc1b, user_id=8cf16c186db04fd7892a4d1381376599, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:21.244126 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e223a-ad43-4def-9d38-c9feefda9e89 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:21.257322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1282/2582] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:21 2022] PUT /identity/v3/projects/436befff7ac84a749cb0945ab6d3bc1b/users/8cf16c186db04fd7892a4d1381376599/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:21.261442 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4703793-c5d6-4f60-95e2-90392e0f3a79 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:21.271906 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4703793-c5d6-4f60-95e2-90392e0f3a79 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:21.329754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0514a-461f-4811-9018-b998c970a98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:21.330363 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0514a-461f-4811-9018-b998c970a98f admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:21.330644 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0514a-461f-4811-9018-b998c970a98f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:21.330919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0514a-461f-4811-9018-b998c970a98f admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:21.331547 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0514a-461f-4811-9018-b998c970a98f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:21.333170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0514a-461f-4811-9018-b998c970a98f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:21.342923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1301/2583] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:21.348066 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e223a-ad43-4def-9d38-c9feefda9e89 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:21.359164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e223a-ad43-4def-9d38-c9feefda9e89 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:21.363896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c8bed-73b9-470e-b9ea-b62c965e36e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:21.364553 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c8bed-73b9-470e-b9ea-b62c965e36e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:21.364838 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c8bed-73b9-470e-b9ea-b62c965e36e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:21.365116 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c8bed-73b9-470e-b9ea-b62c965e36e4 admin admin] PATH_INFO: `/v3/projects/436befff7ac84a749cb0945ab6d3bc1b/users/8cf16c186db04fd7892a4d1381376599/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:21.367019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c8bed-73b9-470e-b9ea-b62c965e36e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=436befff7ac84a749cb0945ab6d3bc1b, user_id=8cf16c186db04fd7892a4d1381376599, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:21.369779 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c8bed-73b9-470e-b9ea-b62c965e36e4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:21.380447 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1283/2584] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:21 2022] PUT /identity/v3/projects/436befff7ac84a749cb0945ab6d3bc1b/users/8cf16c186db04fd7892a4d1381376599/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:21.392463 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8aa00-7a50-4e44-b25c-f815f3caf69c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:21.392951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8aa00-7a50-4e44-b25c-f815f3caf69c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:21.393064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8aa00-7a50-4e44-b25c-f815f3caf69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:21.422604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-2cf8aa00-7a50-4e44-b25c-f815f3caf69c None None] MFA Rules not processed for user `8cf16c186db04fd7892a4d1381376599`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:21.423862 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2cf8aa00-7a50-4e44-b25c-f815f3caf69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:21.432055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:21.494541 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1302/2585] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:21 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:21.505808 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c8bed-73b9-470e-b9ea-b62c965e36e4 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:21.507571 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-fd5c8bed-73b9-470e-b9ea-b62c965e36e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:21.565495 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c8bed-73b9-470e-b9ea-b62c965e36e4 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:21.569361 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafc664-bbc9-4541-bca5-a5abc0d5e422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:21.570008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafc664-bbc9-4541-bca5-a5abc0d5e422 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:21.570287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafc664-bbc9-4541-bca5-a5abc0d5e422 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:21.570561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafc664-bbc9-4541-bca5-a5abc0d5e422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:21.580385 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baafc664-bbc9-4541-bca5-a5abc0d5e422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:21.581814 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baafc664-bbc9-4541-bca5-a5abc0d5e422 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:21.591233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1284/2586] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:21 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:21.740399 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8aa00-7a50-4e44-b25c-f815f3caf69c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:21.749933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8aa00-7a50-4e44-b25c-f815f3caf69c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:21.753977 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c372c-ac02-4a52-82bd-78c9ba3d53fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:21.754596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c372c-ac02-4a52-82bd-78c9ba3d53fb service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:21.754871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c372c-ac02-4a52-82bd-78c9ba3d53fb service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:21.755140 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19c372c-ac02-4a52-82bd-78c9ba3d53fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:21.810302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c372c-ac02-4a52-82bd-78c9ba3d53fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:21.811724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19c372c-ac02-4a52-82bd-78c9ba3d53fb service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:21.821013 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1303/2587] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:21 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:25.587203 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafc664-bbc9-4541-bca5-a5abc0d5e422 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:25.588961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-baafc664-bbc9-4541-bca5-a5abc0d5e422 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:25.656530 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:25.663930 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baafc664-bbc9-4541-bca5-a5abc0d5e422 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:25.669408 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01efb95-9042-4c84-aa47-c5af4d9aef3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:25.670007 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01efb95-9042-4c84-aa47-c5af4d9aef3c service nova] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:25.670283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01efb95-9042-4c84-aa47-c5af4d9aef3c service nova] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:25.670551 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01efb95-9042-4c84-aa47-c5af4d9aef3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:25.672926 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c01efb95-9042-4c84-aa47-c5af4d9aef3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:25.731497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01efb95-9042-4c84-aa47-c5af4d9aef3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:25.733335 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01efb95-9042-4c84-aa47-c5af4d9aef3c service nova] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:25.742453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1285/2588] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:30.275268 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c372c-ac02-4a52-82bd-78c9ba3d53fb service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:30.360058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19c372c-ac02-4a52-82bd-78c9ba3d53fb service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:30.366816 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc834ce-a005-4f9f-9acf-ae988c681d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:30.367648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc834ce-a005-4f9f-9acf-ae988c681d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:30.368113 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc834ce-a005-4f9f-9acf-ae988c681d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:30.368658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc834ce-a005-4f9f-9acf-ae988c681d8f admin admin] PATH_INFO: `/v3/users/4854c2f5c20a494d9c5ef9bb46d8c42e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:30.369914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc834ce-a005-4f9f-9acf-ae988c681d8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4854c2f5c20a494d9c5ef9bb46d8c42e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:30.372153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc834ce-a005-4f9f-9acf-ae988c681d8f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:30.469224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:30.478428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7cc834ce-a005-4f9f-9acf-ae988c681d8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4854c2f5c20a494d9c5ef9bb46d8c42e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:30.502122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7cc834ce-a005-4f9f-9acf-ae988c681d8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4854c2f5c20a494d9c5ef9bb46d8c42e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:30.527496 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7cc834ce-a005-4f9f-9acf-ae988c681d8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4854c2f5c20a494d9c5ef9bb46d8c42e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:30.538058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7cc834ce-a005-4f9f-9acf-ae988c681d8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4854c2f5c20a494d9c5ef9bb46d8c42e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:30.540453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7cc834ce-a005-4f9f-9acf-ae988c681d8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4854c2f5c20a494d9c5ef9bb46d8c42e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:30.550065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1304/2589] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:59:30 2022] DELETE /identity/v3/users/4854c2f5c20a494d9c5ef9bb46d8c42e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:30.554766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47932e-eb7b-414b-8623-ad83bf3eb8c0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:30.555381 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47932e-eb7b-414b-8623-ad83bf3eb8c0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:30.555381 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47932e-eb7b-414b-8623-ad83bf3eb8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:30.583933 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-0d47932e-eb7b-414b-8623-ad83bf3eb8c0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:30.585867 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0d47932e-eb7b-414b-8623-ad83bf3eb8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:30.684674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1286/2590] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:30.696289 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc834ce-a005-4f9f-9acf-ae988c681d8f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:30.705068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7cc834ce-a005-4f9f-9acf-ae988c681d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:30.773786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc834ce-a005-4f9f-9acf-ae988c681d8f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:30.777603 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff29fb7-88c1-4bf0-ac0b-bc11724a4204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:30.778356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff29fb7-88c1-4bf0-ac0b-bc11724a4204 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:30.778697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff29fb7-88c1-4bf0-ac0b-bc11724a4204 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:30.779040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff29fb7-88c1-4bf0-ac0b-bc11724a4204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:30.789390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff29fb7-88c1-4bf0-ac0b-bc11724a4204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:30.790899 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff29fb7-88c1-4bf0-ac0b-bc11724a4204 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:30.802269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1305/2591] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:31.097365 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47932e-eb7b-414b-8623-ad83bf3eb8c0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:31.098847 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-0d47932e-eb7b-414b-8623-ad83bf3eb8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:31.121300 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47932e-eb7b-414b-8623-ad83bf3eb8c0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:31.126612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce8c5a-90af-458b-bbba-ff135157a943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:31.127302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce8c5a-90af-458b-bbba-ff135157a943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:31.127532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce8c5a-90af-458b-bbba-ff135157a943 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:31.127808 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddce8c5a-90af-458b-bbba-ff135157a943 admin admin] PATH_INFO: `/v3/projects/6aa16f68e0fd44449459c2d935e0cf0c` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:31.128970 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce8c5a-90af-458b-bbba-ff135157a943 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aa16f68e0fd44449459c2d935e0cf0c)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:31.130902 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddce8c5a-90af-458b-bbba-ff135157a943 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:31.179587 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ddce8c5a-90af-458b-bbba-ff135157a943 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aa16f68e0fd44449459c2d935e0cf0c', 'request_id': 'req-ddce8c5a-90af-458b-bbba-ff135157a943'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:31.186507 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ddce8c5a-90af-458b-bbba-ff135157a943 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aa16f68e0fd44449459c2d935e0cf0c', 'request_id': 'req-ddce8c5a-90af-458b-bbba-ff135157a943'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:31.195494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ddce8c5a-90af-458b-bbba-ff135157a943 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aa16f68e0fd44449459c2d935e0cf0c', 'request_id': 'req-ddce8c5a-90af-458b-bbba-ff135157a943'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:31.208600 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ddce8c5a-90af-458b-bbba-ff135157a943 admin admin] The token cache is being invalidate because project 6aa16f68e0fd44449459c2d935e0cf0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:59:31.208915 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ddce8c5a-90af-458b-bbba-ff135157a943 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:31.209587 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-ddce8c5a-90af-458b-bbba-ff135157a943 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:31.211211 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1287/2592] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:59:31 2022] DELETE /identity/v3/projects/6aa16f68e0fd44449459c2d935e0cf0c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:31.215337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff29fb7-88c1-4bf0-ac0b-bc11724a4204 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:31.226337 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff29fb7-88c1-4bf0-ac0b-bc11724a4204 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:31.287094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e8220-289f-4d63-b2c2-072e0536f7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:31.287712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e8220-289f-4d63-b2c2-072e0536f7f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:31.288024 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e8220-289f-4d63-b2c2-072e0536f7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:31.288296 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e8220-289f-4d63-b2c2-072e0536f7f1 admin admin] PATH_INFO: `/v3/users/8cf16c186db04fd7892a4d1381376599` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:31.289419 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e8220-289f-4d63-b2c2-072e0536f7f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cf16c186db04fd7892a4d1381376599)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:31.291334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e8220-289f-4d63-b2c2-072e0536f7f1 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:31.341904 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-528e8220-289f-4d63-b2c2-072e0536f7f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cf16c186db04fd7892a4d1381376599'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:31.349243 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-528e8220-289f-4d63-b2c2-072e0536f7f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cf16c186db04fd7892a4d1381376599'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:31.357643 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:31.363539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-528e8220-289f-4d63-b2c2-072e0536f7f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cf16c186db04fd7892a4d1381376599'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:31.373360 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-528e8220-289f-4d63-b2c2-072e0536f7f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cf16c186db04fd7892a4d1381376599'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:31.374191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-528e8220-289f-4d63-b2c2-072e0536f7f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cf16c186db04fd7892a4d1381376599'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:31.384051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1306/2593] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:59:31 2022] DELETE /identity/v3/users/8cf16c186db04fd7892a4d1381376599 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:59:31.775796 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce8c5a-90af-458b-bbba-ff135157a943 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:31.777397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ddce8c5a-90af-458b-bbba-ff135157a943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:31.792305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddce8c5a-90af-458b-bbba-ff135157a943 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:31.822119 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:31.865957 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae7090-4f8e-4336-8659-ac36c2d957d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:31.866600 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae7090-4f8e-4336-8659-ac36c2d957d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:31.866883 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae7090-4f8e-4336-8659-ac36c2d957d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:31.867159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae7090-4f8e-4336-8659-ac36c2d957d6 admin admin] PATH_INFO: `/v3/projects/436befff7ac84a749cb0945ab6d3bc1b` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:31.868331 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae7090-4f8e-4336-8659-ac36c2d957d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=436befff7ac84a749cb0945ab6d3bc1b)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:31.870245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae7090-4f8e-4336-8659-ac36c2d957d6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:31.924208 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-baae7090-4f8e-4336-8659-ac36c2d957d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '436befff7ac84a749cb0945ab6d3bc1b', 'request_id': 'req-baae7090-4f8e-4336-8659-ac36c2d957d6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:31.931116 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-baae7090-4f8e-4336-8659-ac36c2d957d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '436befff7ac84a749cb0945ab6d3bc1b', 'request_id': 'req-baae7090-4f8e-4336-8659-ac36c2d957d6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:31.939470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-baae7090-4f8e-4336-8659-ac36c2d957d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '436befff7ac84a749cb0945ab6d3bc1b', 'request_id': 'req-baae7090-4f8e-4336-8659-ac36c2d957d6'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:31.950683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-baae7090-4f8e-4336-8659-ac36c2d957d6 admin admin] The token cache is being invalidate because project 436befff7ac84a749cb0945ab6d3bc1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:59:31.950969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-baae7090-4f8e-4336-8659-ac36c2d957d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:31.951626 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-baae7090-4f8e-4336-8659-ac36c2d957d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:31.953723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1288/2594] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:59:31 2022] DELETE /identity/v3/projects/436befff7ac84a749cb0945ab6d3bc1b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:31.958743 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf70b9-521f-4132-af40-e84a93bb526d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:31.959065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf70b9-521f-4132-af40-e84a93bb526d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:31.959343 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf70b9-521f-4132-af40-e84a93bb526d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:31.979618 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-47cf70b9-521f-4132-af40-e84a93bb526d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:31.980875 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-47cf70b9-521f-4132-af40-e84a93bb526d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:32.045002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1307/2595] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:59:32.053538 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1b307-454e-4455-9515-9e284d4fe06c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:32.053802 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1b307-454e-4455-9515-9e284d4fe06c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:32.054077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1b307-454e-4455-9515-9e284d4fe06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:32.075021 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d2a1b307-454e-4455-9515-9e284d4fe06c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:32.076291 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d2a1b307-454e-4455-9515-9e284d4fe06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:32.084986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1289/2596] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:32.089173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf70b9-521f-4132-af40-e84a93bb526d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:32.098705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf70b9-521f-4132-af40-e84a93bb526d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:32.103589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4863f-64dc-45fc-9dc9-65126c291a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:32.104257 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4863f-64dc-45fc-9dc9-65126c291a8a admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:32.104534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4863f-64dc-45fc-9dc9-65126c291a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:32.104805 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4863f-64dc-45fc-9dc9-65126c291a8a admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:32.105450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4863f-64dc-45fc-9dc9-65126c291a8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:59:32.106204 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4863f-64dc-45fc-9dc9-65126c291a8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:32.107367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4863f-64dc-45fc-9dc9-65126c291a8a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:32.120662 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1308/2597] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:59:32.125535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1b307-454e-4455-9515-9e284d4fe06c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:32.135244 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1b307-454e-4455-9515-9e284d4fe06c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:32.140837 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35da5c-e5c1-414f-b297-e84db76f1e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:32.141475 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35da5c-e5c1-414f-b297-e84db76f1e25 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:32.141790 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35da5c-e5c1-414f-b297-e84db76f1e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:32.142068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35da5c-e5c1-414f-b297-e84db76f1e25 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:32.142709 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35da5c-e5c1-414f-b297-e84db76f1e25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:32.144681 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35da5c-e5c1-414f-b297-e84db76f1e25 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:32.175172 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1290/2598] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:59:32 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:32.179602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4863f-64dc-45fc-9dc9-65126c291a8a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:32.190834 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4863f-64dc-45fc-9dc9-65126c291a8a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:32.252263 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b21c9-47b2-469c-81b9-1c3e9b5e4501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:32.252878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b21c9-47b2-469c-81b9-1c3e9b5e4501 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:32.253154 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b21c9-47b2-469c-81b9-1c3e9b5e4501 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:32.253423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b21c9-47b2-469c-81b9-1c3e9b5e4501 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:32.254076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b21c9-47b2-469c-81b9-1c3e9b5e4501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:32.256020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b21c9-47b2-469c-81b9-1c3e9b5e4501 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:32.282892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1309/2599] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:59:32 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:59:32.284579 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35da5c-e5c1-414f-b297-e84db76f1e25 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:32.295592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35da5c-e5c1-414f-b297-e84db76f1e25 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:32.300770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe581d1-8ea2-4333-9d59-c4cc9e337611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:32.301449 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe581d1-8ea2-4333-9d59-c4cc9e337611 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:32.301802 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe581d1-8ea2-4333-9d59-c4cc9e337611 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:32.302149 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe581d1-8ea2-4333-9d59-c4cc9e337611 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:32.302871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe581d1-8ea2-4333-9d59-c4cc9e337611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:32.304633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe581d1-8ea2-4333-9d59-c4cc9e337611 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:32.316120 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1291/2600] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:32.320342 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b21c9-47b2-469c-81b9-1c3e9b5e4501 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:32.331785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b21c9-47b2-469c-81b9-1c3e9b5e4501 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:32.336659 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff7c08-9d35-451e-800f-fd6570f1359c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:32.337280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff7c08-9d35-451e-800f-fd6570f1359c admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:32.337556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff7c08-9d35-451e-800f-fd6570f1359c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:32.337832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff7c08-9d35-451e-800f-fd6570f1359c admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:32.338477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff7c08-9d35-451e-800f-fd6570f1359c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:32.340169 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff7c08-9d35-451e-800f-fd6570f1359c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:32.351892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1310/2601] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:32.356376 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe581d1-8ea2-4333-9d59-c4cc9e337611 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:32.370732 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe581d1-8ea2-4333-9d59-c4cc9e337611 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:32.376284 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb7297-d6fa-4d5e-ad27-1f0e41dccd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:32.377077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb7297-d6fa-4d5e-ad27-1f0e41dccd02 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:32.377375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb7297-d6fa-4d5e-ad27-1f0e41dccd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:32.377653 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb7297-d6fa-4d5e-ad27-1f0e41dccd02 admin admin] PATH_INFO: `/v3/projects/0de33fd065c0408dba47516e2ab2f6e4/users/72e31a614ac24152a847bb69a5c9fddc/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:32.394285 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb7297-d6fa-4d5e-ad27-1f0e41dccd02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de33fd065c0408dba47516e2ab2f6e4, user_id=72e31a614ac24152a847bb69a5c9fddc, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:32.397345 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeb7297-d6fa-4d5e-ad27-1f0e41dccd02 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:32.412855 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1292/2602] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:32 2022] PUT /identity/v3/projects/0de33fd065c0408dba47516e2ab2f6e4/users/72e31a614ac24152a847bb69a5c9fddc/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:32.427083 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2206a3e-8b50-45c4-969f-ae9315e72d33 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:32.427394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2206a3e-8b50-45c4-969f-ae9315e72d33 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:32.427676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2206a3e-8b50-45c4-969f-ae9315e72d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:32.461661 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b2206a3e-8b50-45c4-969f-ae9315e72d33 None None] MFA Rules not processed for user `72e31a614ac24152a847bb69a5c9fddc`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:32.462886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b2206a3e-8b50-45c4-969f-ae9315e72d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:32.518048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:32.545827 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1311/2603] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:32.550939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb7297-d6fa-4d5e-ad27-1f0e41dccd02 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:32.612808 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:32.628503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb7297-d6fa-4d5e-ad27-1f0e41dccd02 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:32.635532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5f901-b237-4f38-937c-e7cdb1b0c3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:32.636426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5f901-b237-4f38-937c-e7cdb1b0c3c7 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:32.636737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5f901-b237-4f38-937c-e7cdb1b0c3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:32.637148 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a5f901-b237-4f38-937c-e7cdb1b0c3c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:32.637850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5f901-b237-4f38-937c-e7cdb1b0c3c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:32.640129 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a5f901-b237-4f38-937c-e7cdb1b0c3c7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:32.676045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1293/2604] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:59:32 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:32.676908 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2206a3e-8b50-45c4-969f-ae9315e72d33 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:32.688264 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2206a3e-8b50-45c4-969f-ae9315e72d33 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:32.752185 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25cb50-31b4-41e1-9f45-0311cedda9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:32.752810 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25cb50-31b4-41e1-9f45-0311cedda9d2 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:32.753094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25cb50-31b4-41e1-9f45-0311cedda9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:32.753374 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25cb50-31b4-41e1-9f45-0311cedda9d2 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:32.753996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25cb50-31b4-41e1-9f45-0311cedda9d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:32.755971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25cb50-31b4-41e1-9f45-0311cedda9d2 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:32.778449 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1312/2605] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:59:32 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:59:32.783952 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5f901-b237-4f38-937c-e7cdb1b0c3c7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:32.795017 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a5f901-b237-4f38-937c-e7cdb1b0c3c7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:32.799980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9a4c7-062f-43f0-9d2c-4d71fbca2d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:32.800658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9a4c7-062f-43f0-9d2c-4d71fbca2d93 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:32.800946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9a4c7-062f-43f0-9d2c-4d71fbca2d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:32.801219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9a4c7-062f-43f0-9d2c-4d71fbca2d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:32.801886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9a4c7-062f-43f0-9d2c-4d71fbca2d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:32.803531 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be9a4c7-062f-43f0-9d2c-4d71fbca2d93 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:32.813863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1294/2606] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:32.818159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25cb50-31b4-41e1-9f45-0311cedda9d2 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:32.829143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25cb50-31b4-41e1-9f45-0311cedda9d2 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:32.834832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05dfc9-2957-4360-82c3-a74bb657b0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:32.835498 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05dfc9-2957-4360-82c3-a74bb657b0d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:32.835798 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05dfc9-2957-4360-82c3-a74bb657b0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:32.836150 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05dfc9-2957-4360-82c3-a74bb657b0d0 admin admin] PATH_INFO: `/v3/projects/d3afb7fc0b254b1dbc6cb53ea1f260cf/users/519b5e1852b4471b862f67f1722f14fa/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:32.853237 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05dfc9-2957-4360-82c3-a74bb657b0d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3afb7fc0b254b1dbc6cb53ea1f260cf, user_id=519b5e1852b4471b862f67f1722f14fa, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:32.855942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05dfc9-2957-4360-82c3-a74bb657b0d0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:32.869309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1313/2607] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:32 2022] PUT /identity/v3/projects/d3afb7fc0b254b1dbc6cb53ea1f260cf/users/519b5e1852b4471b862f67f1722f14fa/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:59:32.873321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9a4c7-062f-43f0-9d2c-4d71fbca2d93 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:32.883802 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be9a4c7-062f-43f0-9d2c-4d71fbca2d93 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:32.942214 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85edbdc2-34f5-431d-8388-700d5cfbf4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:32.942829 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edbdc2-34f5-431d-8388-700d5cfbf4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:32.943114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edbdc2-34f5-431d-8388-700d5cfbf4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:32.943390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edbdc2-34f5-431d-8388-700d5cfbf4a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:32.944060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85edbdc2-34f5-431d-8388-700d5cfbf4a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:32.945652 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85edbdc2-34f5-431d-8388-700d5cfbf4a9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:32.955960 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1295/2608] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:32.960166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05dfc9-2957-4360-82c3-a74bb657b0d0 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:32.971198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05dfc9-2957-4360-82c3-a74bb657b0d0 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:32.975992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdf7f1-dd30-47ce-b1b5-f5a8f8143827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:32.976642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdf7f1-dd30-47ce-b1b5-f5a8f8143827 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:32.976927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdf7f1-dd30-47ce-b1b5-f5a8f8143827 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:32.977206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bdf7f1-dd30-47ce-b1b5-f5a8f8143827 admin admin] PATH_INFO: `/v3/projects/d3afb7fc0b254b1dbc6cb53ea1f260cf/users/519b5e1852b4471b862f67f1722f14fa/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:32.979262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdf7f1-dd30-47ce-b1b5-f5a8f8143827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3afb7fc0b254b1dbc6cb53ea1f260cf, user_id=519b5e1852b4471b862f67f1722f14fa, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:32.982086 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bdf7f1-dd30-47ce-b1b5-f5a8f8143827 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:32.992429 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1314/2609] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:32 2022] PUT /identity/v3/projects/d3afb7fc0b254b1dbc6cb53ea1f260cf/users/519b5e1852b4471b862f67f1722f14fa/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:33.002939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff9b13-ea7a-4a48-be3b-41b492f565db None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:33.003225 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff9b13-ea7a-4a48-be3b-41b492f565db None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:33.003503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff9b13-ea7a-4a48-be3b-41b492f565db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:33.032790 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9fff9b13-ea7a-4a48-be3b-41b492f565db None None] MFA Rules not processed for user `519b5e1852b4471b862f67f1722f14fa`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:33.034013 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9fff9b13-ea7a-4a48-be3b-41b492f565db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:33.107084 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1296/2610] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:33.117840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdf7f1-dd30-47ce-b1b5-f5a8f8143827 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:33.119611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-08bdf7f1-dd30-47ce-b1b5-f5a8f8143827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:33.178072 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bdf7f1-dd30-47ce-b1b5-f5a8f8143827 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:33.181777 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c07e8-2eaf-44ba-bacc-c96898d20b15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:33.182643 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c07e8-2eaf-44ba-bacc-c96898d20b15 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:33.182923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c07e8-2eaf-44ba-bacc-c96898d20b15 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:33.183191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c07e8-2eaf-44ba-bacc-c96898d20b15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:33.229796 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:33.248009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c07e8-2eaf-44ba-bacc-c96898d20b15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:33.250426 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c07e8-2eaf-44ba-bacc-c96898d20b15 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:33.262927 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1315/2611] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:35.150766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff9b13-ea7a-4a48-be3b-41b492f565db None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:35.152308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9fff9b13-ea7a-4a48-be3b-41b492f565db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:35.206649 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:35.214081 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff9b13-ea7a-4a48-be3b-41b492f565db None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:35.217848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431242f5-045a-437b-80b2-7b56137aae46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:35.218564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431242f5-045a-437b-80b2-7b56137aae46 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:35.218920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431242f5-045a-437b-80b2-7b56137aae46 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:35.219266 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431242f5-045a-437b-80b2-7b56137aae46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:35.221113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-431242f5-045a-437b-80b2-7b56137aae46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:35.290848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431242f5-045a-437b-80b2-7b56137aae46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:35.292456 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431242f5-045a-437b-80b2-7b56137aae46 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:35.305233 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1297/2612] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:37.588542 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c07e8-2eaf-44ba-bacc-c96898d20b15 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:37.590149 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-191c07e8-2eaf-44ba-bacc-c96898d20b15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:37.602690 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c07e8-2eaf-44ba-bacc-c96898d20b15 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:37.606256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d108a98-4f62-4e6f-a06d-07ed41fb973c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:37.606873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d108a98-4f62-4e6f-a06d-07ed41fb973c service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:37.607146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d108a98-4f62-4e6f-a06d-07ed41fb973c service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:37.607413 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d108a98-4f62-4e6f-a06d-07ed41fb973c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:37.619537 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d108a98-4f62-4e6f-a06d-07ed41fb973c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:37.620992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d108a98-4f62-4e6f-a06d-07ed41fb973c service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:37.640786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1316/2613] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 14:59:38.409069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431242f5-045a-437b-80b2-7b56137aae46 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:38.411508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-431242f5-045a-437b-80b2-7b56137aae46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:38.424536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431242f5-045a-437b-80b2-7b56137aae46 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:38.429297 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4799f-8711-4422-9cef-6b4b51503115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:38.429935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4799f-8711-4422-9cef-6b4b51503115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:38.430215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4799f-8711-4422-9cef-6b4b51503115 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:38.430489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4799f-8711-4422-9cef-6b4b51503115 admin admin] PATH_INFO: `/v3/users/b2c40600d2fd4eea8e4ba999050abd73` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:38.431596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4799f-8711-4422-9cef-6b4b51503115 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2c40600d2fd4eea8e4ba999050abd73)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:38.433810 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4799f-8711-4422-9cef-6b4b51503115 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:38.484654 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-67b4799f-8711-4422-9cef-6b4b51503115 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2c40600d2fd4eea8e4ba999050abd73'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:38.492086 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-67b4799f-8711-4422-9cef-6b4b51503115 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2c40600d2fd4eea8e4ba999050abd73'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:38.504217 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-67b4799f-8711-4422-9cef-6b4b51503115 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2c40600d2fd4eea8e4ba999050abd73'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:38.511737 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-67b4799f-8711-4422-9cef-6b4b51503115 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2c40600d2fd4eea8e4ba999050abd73'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:38.512385 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-67b4799f-8711-4422-9cef-6b4b51503115 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2c40600d2fd4eea8e4ba999050abd73'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:38.518881 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1298/2614] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:59:38 2022] DELETE /identity/v3/users/b2c40600d2fd4eea8e4ba999050abd73 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:38.691122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d108a98-4f62-4e6f-a06d-07ed41fb973c service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:38.703556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d108a98-4f62-4e6f-a06d-07ed41fb973c service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:38.766475 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2f036-470d-4b27-b874-24785c475fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:38.767065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f036-470d-4b27-b874-24785c475fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:38.767352 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f036-470d-4b27-b874-24785c475fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:38.767627 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2f036-470d-4b27-b874-24785c475fe6 admin admin] PATH_INFO: `/v3/projects/db03071b1326441bb9571432be926bf5` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:38.768829 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2f036-470d-4b27-b874-24785c475fe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db03071b1326441bb9571432be926bf5)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:38.770727 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d2f036-470d-4b27-b874-24785c475fe6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:38.819728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-46d2f036-470d-4b27-b874-24785c475fe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db03071b1326441bb9571432be926bf5', 'request_id': 'req-46d2f036-470d-4b27-b874-24785c475fe6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:38.825317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-46d2f036-470d-4b27-b874-24785c475fe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db03071b1326441bb9571432be926bf5', 'request_id': 'req-46d2f036-470d-4b27-b874-24785c475fe6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:38.831362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-46d2f036-470d-4b27-b874-24785c475fe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db03071b1326441bb9571432be926bf5', 'request_id': 'req-46d2f036-470d-4b27-b874-24785c475fe6'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:38.844585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-46d2f036-470d-4b27-b874-24785c475fe6 admin admin] The token cache is being invalidate because project db03071b1326441bb9571432be926bf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:59:38.844945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-46d2f036-470d-4b27-b874-24785c475fe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:38.845606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-46d2f036-470d-4b27-b874-24785c475fe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:38.847068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1317/2615] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:59:38 2022] DELETE /identity/v3/projects/db03071b1326441bb9571432be926bf5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:59:38.852152 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2b8fb-13a8-4408-9d81-9e07ade9516d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:38.852443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2b8fb-13a8-4408-9d81-9e07ade9516d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:38.852720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2b8fb-13a8-4408-9d81-9e07ade9516d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:38.873058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f0e2b8fb-13a8-4408-9d81-9e07ade9516d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:38.874266 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f0e2b8fb-13a8-4408-9d81-9e07ade9516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:38.937646 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1299/2616] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:38.945839 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece0ddc-abc8-4ee1-ab14-b2fa232b4da7 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:38.946130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece0ddc-abc8-4ee1-ab14-b2fa232b4da7 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:38.946410 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece0ddc-abc8-4ee1-ab14-b2fa232b4da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:38.965970 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-dece0ddc-abc8-4ee1-ab14-b2fa232b4da7 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:38.967187 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-dece0ddc-abc8-4ee1-ab14-b2fa232b4da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:38.975934 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1318/2617] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:38.980223 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2b8fb-13a8-4408-9d81-9e07ade9516d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:38.990859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2b8fb-13a8-4408-9d81-9e07ade9516d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:38.995347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3771c-79c3-43a2-9853-1f0b1155bdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:38.995973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3771c-79c3-43a2-9853-1f0b1155bdcc admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:38.996255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3771c-79c3-43a2-9853-1f0b1155bdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:38.996525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3771c-79c3-43a2-9853-1f0b1155bdcc admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:38.997160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3771c-79c3-43a2-9853-1f0b1155bdcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:59:38.997607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3771c-79c3-43a2-9853-1f0b1155bdcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:38.999036 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3771c-79c3-43a2-9853-1f0b1155bdcc admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:39.002074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:39.015877 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1300/2618] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:59:39.017955 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece0ddc-abc8-4ee1-ab14-b2fa232b4da7 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:39.027235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece0ddc-abc8-4ee1-ab14-b2fa232b4da7 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:39.031987 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd3b6a-b26b-4729-9a5f-7988f96da9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:39.032590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd3b6a-b26b-4729-9a5f-7988f96da9dd admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:39.032865 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd3b6a-b26b-4729-9a5f-7988f96da9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:39.033131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd3b6a-b26b-4729-9a5f-7988f96da9dd admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:39.033779 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd3b6a-b26b-4729-9a5f-7988f96da9dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:39.035636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd3b6a-b26b-4729-9a5f-7988f96da9dd admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:39.065604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1319/2619] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:59:39 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:39.069963 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3771c-79c3-43a2-9853-1f0b1155bdcc admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:39.081343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3771c-79c3-43a2-9853-1f0b1155bdcc admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:39.154050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791c1fe-fc19-47d5-a582-f26ddf26cd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:39.154759 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791c1fe-fc19-47d5-a582-f26ddf26cd34 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:39.155135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791c1fe-fc19-47d5-a582-f26ddf26cd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:39.155485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791c1fe-fc19-47d5-a582-f26ddf26cd34 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:39.156788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791c1fe-fc19-47d5-a582-f26ddf26cd34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:39.158778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791c1fe-fc19-47d5-a582-f26ddf26cd34 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:39.188588 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1301/2620] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:59:39 2022] POST /identity/v3/users => generated 284 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:39.192376 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd3b6a-b26b-4729-9a5f-7988f96da9dd admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:39.203866 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd3b6a-b26b-4729-9a5f-7988f96da9dd admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:39.208978 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20278170-686f-4c7b-9ef9-149f33c17588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:39.209737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20278170-686f-4c7b-9ef9-149f33c17588 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:39.210011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20278170-686f-4c7b-9ef9-149f33c17588 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:39.210283 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20278170-686f-4c7b-9ef9-149f33c17588 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:39.210914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20278170-686f-4c7b-9ef9-149f33c17588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:39.212856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20278170-686f-4c7b-9ef9-149f33c17588 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:39.225073 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1320/2621] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 14:59:39.229073 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791c1fe-fc19-47d5-a582-f26ddf26cd34 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:39.240296 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791c1fe-fc19-47d5-a582-f26ddf26cd34 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:39.245693 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15df8e15-dd3a-4e57-9e61-9aaf911e8f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:39.246397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df8e15-dd3a-4e57-9e61-9aaf911e8f0f admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:39.246755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df8e15-dd3a-4e57-9e61-9aaf911e8f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:39.247103 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df8e15-dd3a-4e57-9e61-9aaf911e8f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:39.247848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df8e15-dd3a-4e57-9e61-9aaf911e8f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:39.250158 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15df8e15-dd3a-4e57-9e61-9aaf911e8f0f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:39.267281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1302/2622] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:39.271522 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20278170-686f-4c7b-9ef9-149f33c17588 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:39.282899 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20278170-686f-4c7b-9ef9-149f33c17588 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:39.288516 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3706020-480c-4205-b1a0-b69f2d46cc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:39.289188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3706020-480c-4205-b1a0-b69f2d46cc9c admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:39.289471 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3706020-480c-4205-b1a0-b69f2d46cc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:39.289749 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3706020-480c-4205-b1a0-b69f2d46cc9c admin admin] PATH_INFO: `/v3/projects/b319bb56ab174fea90cb08dbee60afd0/users/7db97030256347a2aa18bfd7bbfe827b/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:39.306152 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3706020-480c-4205-b1a0-b69f2d46cc9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b319bb56ab174fea90cb08dbee60afd0, user_id=7db97030256347a2aa18bfd7bbfe827b, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:39.309256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3706020-480c-4205-b1a0-b69f2d46cc9c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:39.319283 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:39.327640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1321/2623] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:39 2022] PUT /identity/v3/projects/b319bb56ab174fea90cb08dbee60afd0/users/7db97030256347a2aa18bfd7bbfe827b/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:39.338350 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877790a1-ddce-4655-b407-adb55313267b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:39.338646 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877790a1-ddce-4655-b407-adb55313267b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:39.338950 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877790a1-ddce-4655-b407-adb55313267b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:39.371093 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-877790a1-ddce-4655-b407-adb55313267b None None] MFA Rules not processed for user `7db97030256347a2aa18bfd7bbfe827b`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:39.372434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-877790a1-ddce-4655-b407-adb55313267b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:39.442388 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1303/2624] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:39.455082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3706020-480c-4205-b1a0-b69f2d46cc9c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:39.457087 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d3706020-480c-4205-b1a0-b69f2d46cc9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:39.519840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3706020-480c-4205-b1a0-b69f2d46cc9c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:39.524353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1298cf-4e71-4356-83ff-a3dc711ef58f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:39.525102 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1298cf-4e71-4356-83ff-a3dc711ef58f service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:39.526262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1298cf-4e71-4356-83ff-a3dc711ef58f service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:39.526262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1298cf-4e71-4356-83ff-a3dc711ef58f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:39.536162 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1298cf-4e71-4356-83ff-a3dc711ef58f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:39.537641 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1298cf-4e71-4356-83ff-a3dc711ef58f service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:39.546838 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1322/2625] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:42.681269 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877790a1-ddce-4655-b407-adb55313267b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:42.683006 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-877790a1-ddce-4655-b407-adb55313267b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:42.883634 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877790a1-ddce-4655-b407-adb55313267b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:42.888767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df365c1c-5040-4eb6-8ad1-d963b0713778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:42.889420 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df365c1c-5040-4eb6-8ad1-d963b0713778 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:42.889707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df365c1c-5040-4eb6-8ad1-d963b0713778 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:42.890020 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df365c1c-5040-4eb6-8ad1-d963b0713778 admin admin] PATH_INFO: `/v3/users/72e31a614ac24152a847bb69a5c9fddc` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:42.891175 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df365c1c-5040-4eb6-8ad1-d963b0713778 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72e31a614ac24152a847bb69a5c9fddc)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:42.893156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df365c1c-5040-4eb6-8ad1-d963b0713778 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:42.898382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:42.949417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-df365c1c-5040-4eb6-8ad1-d963b0713778 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72e31a614ac24152a847bb69a5c9fddc'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:42.956466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-df365c1c-5040-4eb6-8ad1-d963b0713778 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72e31a614ac24152a847bb69a5c9fddc'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:42.968825 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-df365c1c-5040-4eb6-8ad1-d963b0713778 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72e31a614ac24152a847bb69a5c9fddc'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:42.976639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-df365c1c-5040-4eb6-8ad1-d963b0713778 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72e31a614ac24152a847bb69a5c9fddc'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:42.977487 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-df365c1c-5040-4eb6-8ad1-d963b0713778 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72e31a614ac24152a847bb69a5c9fddc'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:42.983645 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1304/2626] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:59:42 2022] DELETE /identity/v3/users/72e31a614ac24152a847bb69a5c9fddc => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:42.989048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f467f44-d226-4866-9bd7-05d70d95141c None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:42.989553 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f467f44-d226-4866-9bd7-05d70d95141c None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:42.989666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f467f44-d226-4866-9bd7-05d70d95141c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:43.008549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-0f467f44-d226-4866-9bd7-05d70d95141c None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:43.009758 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0f467f44-d226-4866-9bd7-05d70d95141c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:43.069380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1323/2627] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:43.079726 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df365c1c-5040-4eb6-8ad1-d963b0713778 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:43.081221 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-df365c1c-5040-4eb6-8ad1-d963b0713778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:43.138623 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df365c1c-5040-4eb6-8ad1-d963b0713778 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:43.142467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16af69-2628-4056-a559-7540112f5c34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:43.143073 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16af69-2628-4056-a559-7540112f5c34 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:43.143356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16af69-2628-4056-a559-7540112f5c34 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:43.143627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b16af69-2628-4056-a559-7540112f5c34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:43.152675 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16af69-2628-4056-a559-7540112f5c34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:43.154096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b16af69-2628-4056-a559-7540112f5c34 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:43.164074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1305/2628] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:43.480130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f467f44-d226-4866-9bd7-05d70d95141c None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:43.482160 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0f467f44-d226-4866-9bd7-05d70d95141c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:43.498522 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f467f44-d226-4866-9bd7-05d70d95141c None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:43.504524 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:43.505184 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:43.505467 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:43.505814 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424 admin admin] PATH_INFO: `/v3/projects/0de33fd065c0408dba47516e2ab2f6e4` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:43.506918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de33fd065c0408dba47516e2ab2f6e4)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:43.509170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:43.566811 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de33fd065c0408dba47516e2ab2f6e4', 'request_id': 'req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:43.572535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de33fd065c0408dba47516e2ab2f6e4', 'request_id': 'req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:43.579342 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de33fd065c0408dba47516e2ab2f6e4', 'request_id': 'req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:43.591943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424 admin admin] The token cache is being invalidate because project 0de33fd065c0408dba47516e2ab2f6e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:59:43.592240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:43.592920 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:43.595020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1324/2629] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:59:43 2022] DELETE /identity/v3/projects/0de33fd065c0408dba47516e2ab2f6e4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:43.599406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16af69-2628-4056-a559-7540112f5c34 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:43.609774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b16af69-2628-4056-a559-7540112f5c34 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:43.677480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b8dc5-1772-4ccf-9b18-c2187cfe4d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:43.678155 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b8dc5-1772-4ccf-9b18-c2187cfe4d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:43.678326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b8dc5-1772-4ccf-9b18-c2187cfe4d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:43.678756 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b8dc5-1772-4ccf-9b18-c2187cfe4d43 admin admin] PATH_INFO: `/v3/users/519b5e1852b4471b862f67f1722f14fa` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:43.680309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b8dc5-1772-4ccf-9b18-c2187cfe4d43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=519b5e1852b4471b862f67f1722f14fa)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:43.683408 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b8dc5-1772-4ccf-9b18-c2187cfe4d43 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:43.722920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:43.746888 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-313b8dc5-1772-4ccf-9b18-c2187cfe4d43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '519b5e1852b4471b862f67f1722f14fa'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:43.754761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-313b8dc5-1772-4ccf-9b18-c2187cfe4d43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '519b5e1852b4471b862f67f1722f14fa'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:43.767639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-313b8dc5-1772-4ccf-9b18-c2187cfe4d43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '519b5e1852b4471b862f67f1722f14fa'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:43.781162 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-313b8dc5-1772-4ccf-9b18-c2187cfe4d43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '519b5e1852b4471b862f67f1722f14fa'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:43.781845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-313b8dc5-1772-4ccf-9b18-c2187cfe4d43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '519b5e1852b4471b862f67f1722f14fa'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:43.788912 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1306/2630] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:59:43 2022] DELETE /identity/v3/users/519b5e1852b4471b862f67f1722f14fa => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:44.151343 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:44.152926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:44.167435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37fd47-2fc2-4f19-a8d6-ad7936d78424 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:44.196946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:44.234585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a172276-3450-4d8a-8a53-40166ebb0a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:44.235233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a172276-3450-4d8a-8a53-40166ebb0a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:44.235514 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a172276-3450-4d8a-8a53-40166ebb0a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:44.235787 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a172276-3450-4d8a-8a53-40166ebb0a83 admin admin] PATH_INFO: `/v3/projects/d3afb7fc0b254b1dbc6cb53ea1f260cf` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:44.236954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a172276-3450-4d8a-8a53-40166ebb0a83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3afb7fc0b254b1dbc6cb53ea1f260cf)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:44.238862 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a172276-3450-4d8a-8a53-40166ebb0a83 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:44.288879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6a172276-3450-4d8a-8a53-40166ebb0a83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3afb7fc0b254b1dbc6cb53ea1f260cf', 'request_id': 'req-6a172276-3450-4d8a-8a53-40166ebb0a83'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:44.294589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6a172276-3450-4d8a-8a53-40166ebb0a83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3afb7fc0b254b1dbc6cb53ea1f260cf', 'request_id': 'req-6a172276-3450-4d8a-8a53-40166ebb0a83'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:44.306720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6a172276-3450-4d8a-8a53-40166ebb0a83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3afb7fc0b254b1dbc6cb53ea1f260cf', 'request_id': 'req-6a172276-3450-4d8a-8a53-40166ebb0a83'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:44.315435 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6a172276-3450-4d8a-8a53-40166ebb0a83 admin admin] The token cache is being invalidate because project d3afb7fc0b254b1dbc6cb53ea1f260cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:59:44.315734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6a172276-3450-4d8a-8a53-40166ebb0a83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:44.316413 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6a172276-3450-4d8a-8a53-40166ebb0a83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:44.317949 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1325/2631] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:59:44 2022] DELETE /identity/v3/projects/d3afb7fc0b254b1dbc6cb53ea1f260cf => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:44.324020 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665968df-93c7-4c31-b7c1-705a1da16725 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:44.324314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665968df-93c7-4c31-b7c1-705a1da16725 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:44.324597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665968df-93c7-4c31-b7c1-705a1da16725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:44.344915 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-665968df-93c7-4c31-b7c1-705a1da16725 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:44.346277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-665968df-93c7-4c31-b7c1-705a1da16725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:44.412502 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1307/2632] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:44.421897 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eb281-47e9-449e-97c0-1bbb1a17b9d5 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:44.422274 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eb281-47e9-449e-97c0-1bbb1a17b9d5 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:44.422558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eb281-47e9-449e-97c0-1bbb1a17b9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:44.445384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-861eb281-47e9-449e-97c0-1bbb1a17b9d5 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:44.446594 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-861eb281-47e9-449e-97c0-1bbb1a17b9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:44.456219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1326/2633] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:44.460584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665968df-93c7-4c31-b7c1-705a1da16725 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:44.470459 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665968df-93c7-4c31-b7c1-705a1da16725 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:44.475324 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f805e-ab69-414f-9943-917418bdfa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:44.475971 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f805e-ab69-414f-9943-917418bdfa09 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:44.476255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f805e-ab69-414f-9943-917418bdfa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:44.476529 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f805e-ab69-414f-9943-917418bdfa09 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:44.477177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f805e-ab69-414f-9943-917418bdfa09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:59:44.477474 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f805e-ab69-414f-9943-917418bdfa09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:44.479095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f805e-ab69-414f-9943-917418bdfa09 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:44.491597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1308/2634] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:59:44.496153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861eb281-47e9-449e-97c0-1bbb1a17b9d5 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:44.506139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861eb281-47e9-449e-97c0-1bbb1a17b9d5 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:44.510810 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fdf79-0839-4f4d-92a3-3489b12acb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:44.511422 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fdf79-0839-4f4d-92a3-3489b12acb06 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:44.511702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fdf79-0839-4f4d-92a3-3489b12acb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:44.512010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fdf79-0839-4f4d-92a3-3489b12acb06 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:44.512637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fdf79-0839-4f4d-92a3-3489b12acb06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:44.514512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fdf79-0839-4f4d-92a3-3489b12acb06 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:44.552993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1327/2635] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:59:44 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:44.557635 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f805e-ab69-414f-9943-917418bdfa09 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:44.568820 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f805e-ab69-414f-9943-917418bdfa09 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:44.628429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c936bf-86a7-43d3-aac5-54fac27de5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:44.629164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c936bf-86a7-43d3-aac5-54fac27de5cc admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:44.629478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c936bf-86a7-43d3-aac5-54fac27de5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:44.629751 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c936bf-86a7-43d3-aac5-54fac27de5cc admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:44.630385 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c936bf-86a7-43d3-aac5-54fac27de5cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:44.632349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c936bf-86a7-43d3-aac5-54fac27de5cc admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:44.655391 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1309/2636] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:59:44 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:44.660948 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fdf79-0839-4f4d-92a3-3489b12acb06 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:44.673364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fdf79-0839-4f4d-92a3-3489b12acb06 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:44.678273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ce1d1-3ea3-4360-bb12-e331a58e6306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:44.678893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ce1d1-3ea3-4360-bb12-e331a58e6306 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:44.679175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ce1d1-3ea3-4360-bb12-e331a58e6306 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:44.679448 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ce1d1-3ea3-4360-bb12-e331a58e6306 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:44.680222 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ce1d1-3ea3-4360-bb12-e331a58e6306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:44.681827 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ce1d1-3ea3-4360-bb12-e331a58e6306 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:44.693879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1328/2637] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:44.698234 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c936bf-86a7-43d3-aac5-54fac27de5cc admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:44.708932 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c936bf-86a7-43d3-aac5-54fac27de5cc admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:44.715609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2a7da-4a95-4946-a209-4f638687eda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:44.716496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2a7da-4a95-4946-a209-4f638687eda7 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:44.716944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2a7da-4a95-4946-a209-4f638687eda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:44.717389 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2a7da-4a95-4946-a209-4f638687eda7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:44.718829 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2a7da-4a95-4946-a209-4f638687eda7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:44.720919 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2a7da-4a95-4946-a209-4f638687eda7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:44.734850 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1310/2638] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:44.748505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ce1d1-3ea3-4360-bb12-e331a58e6306 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:44.753532 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ce1d1-3ea3-4360-bb12-e331a58e6306 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:44.758547 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc69d56-328a-47e0-83b7-840e1f374b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:44.759271 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc69d56-328a-47e0-83b7-840e1f374b48 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:44.759546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc69d56-328a-47e0-83b7-840e1f374b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:44.760124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc69d56-328a-47e0-83b7-840e1f374b48 admin admin] PATH_INFO: `/v3/projects/4502d9a0ff3a414f855bf0beb35ffd35/users/0edc61e9f06e47e69fad8ba5e2a7f201/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:44.906360 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc69d56-328a-47e0-83b7-840e1f374b48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4502d9a0ff3a414f855bf0beb35ffd35, user_id=0edc61e9f06e47e69fad8ba5e2a7f201, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:44.909732 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc69d56-328a-47e0-83b7-840e1f374b48 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:44.924939 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1329/2639] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:44 2022] PUT /identity/v3/projects/4502d9a0ff3a414f855bf0beb35ffd35/users/0edc61e9f06e47e69fad8ba5e2a7f201/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:44.930718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b559d-36d6-4195-bdaf-cee3b5c4c0ef None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:44.931132 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b559d-36d6-4195-bdaf-cee3b5c4c0ef None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:44.931284 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b559d-36d6-4195-bdaf-cee3b5c4c0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:44.950992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c45b559d-36d6-4195-bdaf-cee3b5c4c0ef None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:44.952890 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c45b559d-36d6-4195-bdaf-cee3b5c4c0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:44.958762 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:45.028462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1311/2640] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:45.039379 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc69d56-328a-47e0-83b7-840e1f374b48 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:45.041182 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9cc69d56-328a-47e0-83b7-840e1f374b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:45.111194 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc69d56-328a-47e0-83b7-840e1f374b48 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:45.117791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666fef7-a938-42be-90f3-9ea210a78d5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:45.118372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666fef7-a938-42be-90f3-9ea210a78d5d service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:45.118654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666fef7-a938-42be-90f3-9ea210a78d5d service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:45.118926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666fef7-a938-42be-90f3-9ea210a78d5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:45.127432 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:45.130621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666fef7-a938-42be-90f3-9ea210a78d5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:45.133054 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666fef7-a938-42be-90f3-9ea210a78d5d service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:45.152754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1330/2641] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:45.876554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42811c-6dff-470e-868d-320df44d0135 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:45.877109 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42811c-6dff-470e-868d-320df44d0135 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:45.877212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42811c-6dff-470e-868d-320df44d0135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:45.908251 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ed42811c-6dff-470e-868d-320df44d0135 None None] MFA Rules not processed for user `0edc61e9f06e47e69fad8ba5e2a7f201`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:45.909476 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ed42811c-6dff-470e-868d-320df44d0135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:45.983045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1312/2642] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:45.992655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666fef7-a938-42be-90f3-9ea210a78d5d service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:45.994628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9666fef7-a938-42be-90f3-9ea210a78d5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:46.077290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666fef7-a938-42be-90f3-9ea210a78d5d service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:46.082640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814bdb7-5098-46c2-bdcb-901370d1e2ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:46.083342 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814bdb7-5098-46c2-bdcb-901370d1e2ea service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:46.083694 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814bdb7-5098-46c2-bdcb-901370d1e2ea service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:46.084061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814bdb7-5098-46c2-bdcb-901370d1e2ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:46.105194 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814bdb7-5098-46c2-bdcb-901370d1e2ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:46.106722 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814bdb7-5098-46c2-bdcb-901370d1e2ea service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:46.117386 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1331/2643] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:47.673168 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183cef8-3bda-471c-bf77-c6cd2843204f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:47.673678 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183cef8-3bda-471c-bf77-c6cd2843204f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:47.673778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183cef8-3bda-471c-bf77-c6cd2843204f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:47.693343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-5183cef8-3bda-471c-bf77-c6cd2843204f None None] MFA Rules not processed for user `0edc61e9f06e47e69fad8ba5e2a7f201`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:47.694584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5183cef8-3bda-471c-bf77-c6cd2843204f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:47.701406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1313/2644] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:48.337791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814bdb7-5098-46c2-bdcb-901370d1e2ea service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:48.339417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9814bdb7-5098-46c2-bdcb-901370d1e2ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:48.358292 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814bdb7-5098-46c2-bdcb-901370d1e2ea service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:48.363031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220ab51-06ba-4e94-ae42-41e3829b25a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:48.363750 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220ab51-06ba-4e94-ae42-41e3829b25a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:48.364763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220ab51-06ba-4e94-ae42-41e3829b25a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:48.365104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220ab51-06ba-4e94-ae42-41e3829b25a0 admin admin] PATH_INFO: `/v3/users/7db97030256347a2aa18bfd7bbfe827b` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:48.366349 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220ab51-06ba-4e94-ae42-41e3829b25a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7db97030256347a2aa18bfd7bbfe827b)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:48.368501 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220ab51-06ba-4e94-ae42-41e3829b25a0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:48.427161 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0220ab51-06ba-4e94-ae42-41e3829b25a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db97030256347a2aa18bfd7bbfe827b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:48.436393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0220ab51-06ba-4e94-ae42-41e3829b25a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db97030256347a2aa18bfd7bbfe827b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:48.462428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0220ab51-06ba-4e94-ae42-41e3829b25a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db97030256347a2aa18bfd7bbfe827b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:48.467198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0220ab51-06ba-4e94-ae42-41e3829b25a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db97030256347a2aa18bfd7bbfe827b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:48.467993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-0220ab51-06ba-4e94-ae42-41e3829b25a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db97030256347a2aa18bfd7bbfe827b'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:48.474135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1332/2645] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:59:48 2022] DELETE /identity/v3/users/7db97030256347a2aa18bfd7bbfe827b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:48.478920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28137f-3afe-4d5c-ae49-712b1279e0a4 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:48.479200 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28137f-3afe-4d5c-ae49-712b1279e0a4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:48.479467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28137f-3afe-4d5c-ae49-712b1279e0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:48.501230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-dc28137f-3afe-4d5c-ae49-712b1279e0a4 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:48.502663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dc28137f-3afe-4d5c-ae49-712b1279e0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:48.569864 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1314/2646] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:48.580288 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220ab51-06ba-4e94-ae42-41e3829b25a0 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:48.582375 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-0220ab51-06ba-4e94-ae42-41e3829b25a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:48.643659 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220ab51-06ba-4e94-ae42-41e3829b25a0 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:48.647649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf4e80b-1b7d-4bd2-b419-da72ae6d7f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:48.648365 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4e80b-1b7d-4bd2-b419-da72ae6d7f56 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:48.648653 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4e80b-1b7d-4bd2-b419-da72ae6d7f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:48.648910 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf4e80b-1b7d-4bd2-b419-da72ae6d7f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:48.662260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4e80b-1b7d-4bd2-b419-da72ae6d7f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:48.663999 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf4e80b-1b7d-4bd2-b419-da72ae6d7f56 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:48.674631 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1333/2647] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:49.150100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28137f-3afe-4d5c-ae49-712b1279e0a4 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:49.151975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dc28137f-3afe-4d5c-ae49-712b1279e0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:49.166250 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28137f-3afe-4d5c-ae49-712b1279e0a4 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:49.172288 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86094397-8351-428f-b5e6-c3cc755f4167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:49.173136 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86094397-8351-428f-b5e6-c3cc755f4167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:49.173500 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86094397-8351-428f-b5e6-c3cc755f4167 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:49.174073 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86094397-8351-428f-b5e6-c3cc755f4167 admin admin] PATH_INFO: `/v3/projects/b319bb56ab174fea90cb08dbee60afd0` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:49.175938 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86094397-8351-428f-b5e6-c3cc755f4167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b319bb56ab174fea90cb08dbee60afd0)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:49.178449 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86094397-8351-428f-b5e6-c3cc755f4167 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:49.222065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:49.240476 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-86094397-8351-428f-b5e6-c3cc755f4167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b319bb56ab174fea90cb08dbee60afd0', 'request_id': 'req-86094397-8351-428f-b5e6-c3cc755f4167'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:49.247416 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-86094397-8351-428f-b5e6-c3cc755f4167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b319bb56ab174fea90cb08dbee60afd0', 'request_id': 'req-86094397-8351-428f-b5e6-c3cc755f4167'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:49.256946 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-86094397-8351-428f-b5e6-c3cc755f4167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b319bb56ab174fea90cb08dbee60afd0', 'request_id': 'req-86094397-8351-428f-b5e6-c3cc755f4167'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:49.269606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-86094397-8351-428f-b5e6-c3cc755f4167 admin admin] The token cache is being invalidate because project b319bb56ab174fea90cb08dbee60afd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:59:49.269899 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-86094397-8351-428f-b5e6-c3cc755f4167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:49.270561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-86094397-8351-428f-b5e6-c3cc755f4167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:49.272505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1315/2648] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:59:49 2022] DELETE /identity/v3/projects/b319bb56ab174fea90cb08dbee60afd0 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:49.279449 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fce54-60f0-4969-a574-3e134e6c5e03 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:49.280140 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fce54-60f0-4969-a574-3e134e6c5e03 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:49.281651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2fce54-60f0-4969-a574-3e134e6c5e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:49.307727 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-de2fce54-60f0-4969-a574-3e134e6c5e03 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:49.309033 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-de2fce54-60f0-4969-a574-3e134e6c5e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:49.319810 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:49.383774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1334/2649] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:59:49.392935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a9fb9-8911-4518-a6a9-54b5dfbfbf1a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:49.393251 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a9fb9-8911-4518-a6a9-54b5dfbfbf1a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:49.393533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a9fb9-8911-4518-a6a9-54b5dfbfbf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:49.415435 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-182a9fb9-8911-4518-a6a9-54b5dfbfbf1a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:49.416807 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-182a9fb9-8911-4518-a6a9-54b5dfbfbf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:49.427880 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1316/2650] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 14:59:49.430863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fce54-60f0-4969-a574-3e134e6c5e03 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:49.446304 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2fce54-60f0-4969-a574-3e134e6c5e03 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:49.460150 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe66bc8-3e89-44a8-8e6d-0ced4371589a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:49.461180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe66bc8-3e89-44a8-8e6d-0ced4371589a admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:49.461764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe66bc8-3e89-44a8-8e6d-0ced4371589a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:49.462243 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe66bc8-3e89-44a8-8e6d-0ced4371589a admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:49.463102 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe66bc8-3e89-44a8-8e6d-0ced4371589a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:59:49.463624 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe66bc8-3e89-44a8-8e6d-0ced4371589a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:49.466122 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe66bc8-3e89-44a8-8e6d-0ced4371589a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:49.481071 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1335/2651] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:59:49.485371 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a9fb9-8911-4518-a6a9-54b5dfbfbf1a None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:49.495787 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a9fb9-8911-4518-a6a9-54b5dfbfbf1a None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:49.501442 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc000b-e8cd-4313-9e69-ef6fb0fa6d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:49.502109 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc000b-e8cd-4313-9e69-ef6fb0fa6d6b admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:49.502389 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc000b-e8cd-4313-9e69-ef6fb0fa6d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:49.502674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc000b-e8cd-4313-9e69-ef6fb0fa6d6b admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:49.503304 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc000b-e8cd-4313-9e69-ef6fb0fa6d6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:49.506107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc000b-e8cd-4313-9e69-ef6fb0fa6d6b admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:49.540777 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1317/2652] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:59:49 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:49.545064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe66bc8-3e89-44a8-8e6d-0ced4371589a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:49.555561 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe66bc8-3e89-44a8-8e6d-0ced4371589a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:49.620350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f2e2c-f1b5-4b7f-82b6-722f197103fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:49.620983 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f2e2c-f1b5-4b7f-82b6-722f197103fd admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:49.621266 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f2e2c-f1b5-4b7f-82b6-722f197103fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:49.621543 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f2e2c-f1b5-4b7f-82b6-722f197103fd admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:49.622198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f2e2c-f1b5-4b7f-82b6-722f197103fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:49.624324 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2f2e2c-f1b5-4b7f-82b6-722f197103fd admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:49.650453 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1336/2653] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:59:49 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:49.654703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc000b-e8cd-4313-9e69-ef6fb0fa6d6b admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:49.665975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc000b-e8cd-4313-9e69-ef6fb0fa6d6b admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:49.670805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bee8c-15de-4488-a370-df698c6cde0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:49.672033 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bee8c-15de-4488-a370-df698c6cde0e admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:49.672310 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bee8c-15de-4488-a370-df698c6cde0e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:49.672593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274bee8c-15de-4488-a370-df698c6cde0e admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:49.673221 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274bee8c-15de-4488-a370-df698c6cde0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:49.674801 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274bee8c-15de-4488-a370-df698c6cde0e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:49.688060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1318/2654] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:49.693507 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f2e2c-f1b5-4b7f-82b6-722f197103fd admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:49.704414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2f2e2c-f1b5-4b7f-82b6-722f197103fd admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:49.710510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76afae9a-98f5-4def-a60e-572dd7d72339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:49.710510 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afae9a-98f5-4def-a60e-572dd7d72339 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:49.710804 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afae9a-98f5-4def-a60e-572dd7d72339 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:49.711037 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afae9a-98f5-4def-a60e-572dd7d72339 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:49.711675 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afae9a-98f5-4def-a60e-572dd7d72339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:49.713302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76afae9a-98f5-4def-a60e-572dd7d72339 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:49.727111 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1337/2655] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:49.728330 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bee8c-15de-4488-a370-df698c6cde0e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:49.746933 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274bee8c-15de-4488-a370-df698c6cde0e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:49.752811 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2852c5-7096-4d33-99d5-844aa3bad979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:49.753543 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2852c5-7096-4d33-99d5-844aa3bad979 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:49.754661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2852c5-7096-4d33-99d5-844aa3bad979 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:49.754661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2852c5-7096-4d33-99d5-844aa3bad979 admin admin] PATH_INFO: `/v3/projects/0cf29504a31e42e384c6318ae4be9e7a/users/bfbabfee5d3a47a0a273d1e131589b6a/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:49.773750 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2852c5-7096-4d33-99d5-844aa3bad979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf29504a31e42e384c6318ae4be9e7a, user_id=bfbabfee5d3a47a0a273d1e131589b6a, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:49.776554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2852c5-7096-4d33-99d5-844aa3bad979 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:49.793691 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1319/2656] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:49 2022] PUT /identity/v3/projects/0cf29504a31e42e384c6318ae4be9e7a/users/bfbabfee5d3a47a0a273d1e131589b6a/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:49.804261 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278bafb-1b9b-40c6-9ced-f4ad197a7343 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:49.804546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278bafb-1b9b-40c6-9ced-f4ad197a7343 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:49.804834 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278bafb-1b9b-40c6-9ced-f4ad197a7343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:49.837064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e278bafb-1b9b-40c6-9ced-f4ad197a7343 None None] MFA Rules not processed for user `bfbabfee5d3a47a0a273d1e131589b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:49.838458 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e278bafb-1b9b-40c6-9ced-f4ad197a7343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:49.911060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1338/2657] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:49.915308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2852c5-7096-4d33-99d5-844aa3bad979 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:49.980703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2852c5-7096-4d33-99d5-844aa3bad979 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:49.985822 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d341d0-8bfe-4df0-90fb-3c142150c781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:49.986319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d341d0-8bfe-4df0-90fb-3c142150c781 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:49.986753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d341d0-8bfe-4df0-90fb-3c142150c781 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:49.987066 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d341d0-8bfe-4df0-90fb-3c142150c781 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:49.988616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d341d0-8bfe-4df0-90fb-3c142150c781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:49.990014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d341d0-8bfe-4df0-90fb-3c142150c781 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:50.019558 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1320/2658] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:59:49 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 14:59:50.024859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278bafb-1b9b-40c6-9ced-f4ad197a7343 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:50.036575 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278bafb-1b9b-40c6-9ced-f4ad197a7343 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:50.043126 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:50.107914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4cda05-4493-41d1-b772-41b56dcfd812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:50.109106 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cda05-4493-41d1-b772-41b56dcfd812 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:50.109385 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cda05-4493-41d1-b772-41b56dcfd812 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:50.109651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4cda05-4493-41d1-b772-41b56dcfd812 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:50.110309 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4cda05-4493-41d1-b772-41b56dcfd812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:50.112727 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4cda05-4493-41d1-b772-41b56dcfd812 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:50.136628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1339/2659] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:59:50 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:59:50.140706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d341d0-8bfe-4df0-90fb-3c142150c781 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:50.152161 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d341d0-8bfe-4df0-90fb-3c142150c781 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:50.161411 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14ed5d-b113-4acd-a31e-b0691848f2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:50.162383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14ed5d-b113-4acd-a31e-b0691848f2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:50.162933 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14ed5d-b113-4acd-a31e-b0691848f2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:50.163587 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14ed5d-b113-4acd-a31e-b0691848f2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:50.164301 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14ed5d-b113-4acd-a31e-b0691848f2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:50.166257 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14ed5d-b113-4acd-a31e-b0691848f2c0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:50.179980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1321/2660] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:50.184566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4cda05-4493-41d1-b772-41b56dcfd812 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:50.197007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4cda05-4493-41d1-b772-41b56dcfd812 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:50.203986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2c9be-241c-446e-8096-d0345455a2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:50.204715 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2c9be-241c-446e-8096-d0345455a2dc admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:50.205026 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2c9be-241c-446e-8096-d0345455a2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:50.205334 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2c9be-241c-446e-8096-d0345455a2dc admin admin] PATH_INFO: `/v3/projects/e5925a31c6e14e899aa90fb855bb02e2/users/350d31ccdb02403c857a2e69a1c83bb5/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:50.222783 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2c9be-241c-446e-8096-d0345455a2dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5925a31c6e14e899aa90fb855bb02e2, user_id=350d31ccdb02403c857a2e69a1c83bb5, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:50.225569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2c9be-241c-446e-8096-d0345455a2dc admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:50.244610 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1340/2661] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:50 2022] PUT /identity/v3/projects/e5925a31c6e14e899aa90fb855bb02e2/users/350d31ccdb02403c857a2e69a1c83bb5/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 14:59:50.248556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14ed5d-b113-4acd-a31e-b0691848f2c0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:50.261149 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14ed5d-b113-4acd-a31e-b0691848f2c0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:50.295097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:50.373368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5c268-616a-469b-b92c-3630ac799edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:50.374967 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5c268-616a-469b-b92c-3630ac799edb admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:50.375422 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5c268-616a-469b-b92c-3630ac799edb admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:50.375877 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5c268-616a-469b-b92c-3630ac799edb admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:50.376705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5c268-616a-469b-b92c-3630ac799edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:50.378730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d5c268-616a-469b-b92c-3630ac799edb admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:50.394550 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1322/2662] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:50.398454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2c9be-241c-446e-8096-d0345455a2dc admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:50.415966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2c9be-241c-446e-8096-d0345455a2dc admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:50.422911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064b32a-e527-4082-8411-527895c36001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:50.423696 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064b32a-e527-4082-8411-527895c36001 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:50.424101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064b32a-e527-4082-8411-527895c36001 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:50.424555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064b32a-e527-4082-8411-527895c36001 admin admin] PATH_INFO: `/v3/projects/e5925a31c6e14e899aa90fb855bb02e2/users/350d31ccdb02403c857a2e69a1c83bb5/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:50.427208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064b32a-e527-4082-8411-527895c36001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5925a31c6e14e899aa90fb855bb02e2, user_id=350d31ccdb02403c857a2e69a1c83bb5, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:50.430974 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064b32a-e527-4082-8411-527895c36001 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:50.456747 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1341/2663] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:50 2022] PUT /identity/v3/projects/e5925a31c6e14e899aa90fb855bb02e2/users/350d31ccdb02403c857a2e69a1c83bb5/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:50.458402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f7bb0-8243-44de-90b9-99504f74c529 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:50.458575 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f7bb0-8243-44de-90b9-99504f74c529 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:50.458799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f7bb0-8243-44de-90b9-99504f74c529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:50.492859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-550f7bb0-8243-44de-90b9-99504f74c529 None None] MFA Rules not processed for user `350d31ccdb02403c857a2e69a1c83bb5`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:50.494220 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-550f7bb0-8243-44de-90b9-99504f74c529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:50.582885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1323/2664] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:50.594328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064b32a-e527-4082-8411-527895c36001 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:50.599314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f064b32a-e527-4082-8411-527895c36001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:50.672625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064b32a-e527-4082-8411-527895c36001 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:50.676695 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f8e04-d66e-44a7-8744-92088ac67563 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:50.677449 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f8e04-d66e-44a7-8744-92088ac67563 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:50.677724 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f8e04-d66e-44a7-8744-92088ac67563 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:50.677992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f8e04-d66e-44a7-8744-92088ac67563 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:50.689621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f8e04-d66e-44a7-8744-92088ac67563 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:50.696689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f8e04-d66e-44a7-8744-92088ac67563 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:50.709151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1342/2665] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:51.479116 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f7bb0-8243-44de-90b9-99504f74c529 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:51.584006 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f7bb0-8243-44de-90b9-99504f74c529 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:51.593884 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7a58c-9c37-4594-9ffa-f6a67bd8785f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:51.595082 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7a58c-9c37-4594-9ffa-f6a67bd8785f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:51.595082 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7a58c-9c37-4594-9ffa-f6a67bd8785f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:51.595082 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7a58c-9c37-4594-9ffa-f6a67bd8785f admin admin] PATH_INFO: `/v3/users/bfbabfee5d3a47a0a273d1e131589b6a` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:51.596525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7a58c-9c37-4594-9ffa-f6a67bd8785f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfbabfee5d3a47a0a273d1e131589b6a)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:51.598461 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7a58c-9c37-4594-9ffa-f6a67bd8785f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:51.656683 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-55f7a58c-9c37-4594-9ffa-f6a67bd8785f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfbabfee5d3a47a0a273d1e131589b6a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:51.662289 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:51.669057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-55f7a58c-9c37-4594-9ffa-f6a67bd8785f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfbabfee5d3a47a0a273d1e131589b6a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:51.690600 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-55f7a58c-9c37-4594-9ffa-f6a67bd8785f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfbabfee5d3a47a0a273d1e131589b6a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:51.709601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-55f7a58c-9c37-4594-9ffa-f6a67bd8785f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfbabfee5d3a47a0a273d1e131589b6a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:51.710246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-55f7a58c-9c37-4594-9ffa-f6a67bd8785f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfbabfee5d3a47a0a273d1e131589b6a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:51.726635 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1324/2666] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:59:51 2022] DELETE /identity/v3/users/bfbabfee5d3a47a0a273d1e131589b6a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:51.731265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bb576-d2ec-439b-b798-ec018520e8c0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:51.732020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bb576-d2ec-439b-b798-ec018520e8c0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:51.732410 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910bb576-d2ec-439b-b798-ec018520e8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:51.761147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-910bb576-d2ec-439b-b798-ec018520e8c0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:51.762794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-910bb576-d2ec-439b-b798-ec018520e8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:51.901697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1343/2667] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:51.916097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7a58c-9c37-4594-9ffa-f6a67bd8785f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:51.917975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-55f7a58c-9c37-4594-9ffa-f6a67bd8785f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:52.041643 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7a58c-9c37-4594-9ffa-f6a67bd8785f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:52.046492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8972cb-d014-4cb0-a6b3-69fda738f396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:52.047147 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8972cb-d014-4cb0-a6b3-69fda738f396 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:52.047432 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8972cb-d014-4cb0-a6b3-69fda738f396 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:52.047704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8972cb-d014-4cb0-a6b3-69fda738f396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:52.057861 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8972cb-d014-4cb0-a6b3-69fda738f396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:52.059308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8972cb-d014-4cb0-a6b3-69fda738f396 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:52.069486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1325/2668] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:52.349585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bb576-d2ec-439b-b798-ec018520e8c0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:52.351059 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-910bb576-d2ec-439b-b798-ec018520e8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:52.360417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:52.363952 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910bb576-d2ec-439b-b798-ec018520e8c0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:52.369494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92caa8-4470-452a-9804-d9f4095624e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:52.371410 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92caa8-4470-452a-9804-d9f4095624e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:52.372288 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92caa8-4470-452a-9804-d9f4095624e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:52.373065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92caa8-4470-452a-9804-d9f4095624e5 admin admin] PATH_INFO: `/v3/projects/0cf29504a31e42e384c6318ae4be9e7a` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:52.375593 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92caa8-4470-452a-9804-d9f4095624e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cf29504a31e42e384c6318ae4be9e7a)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:52.379475 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92caa8-4470-452a-9804-d9f4095624e5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:52.440524 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4b92caa8-4470-452a-9804-d9f4095624e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cf29504a31e42e384c6318ae4be9e7a', 'request_id': 'req-4b92caa8-4470-452a-9804-d9f4095624e5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:52.447132 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4b92caa8-4470-452a-9804-d9f4095624e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cf29504a31e42e384c6318ae4be9e7a', 'request_id': 'req-4b92caa8-4470-452a-9804-d9f4095624e5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:52.454044 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4b92caa8-4470-452a-9804-d9f4095624e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cf29504a31e42e384c6318ae4be9e7a', 'request_id': 'req-4b92caa8-4470-452a-9804-d9f4095624e5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:52.469308 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4b92caa8-4470-452a-9804-d9f4095624e5 admin admin] The token cache is being invalidate because project 0cf29504a31e42e384c6318ae4be9e7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:59:52.469602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4b92caa8-4470-452a-9804-d9f4095624e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:52.470260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-4b92caa8-4470-452a-9804-d9f4095624e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:52.471966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1344/2669] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:59:52 2022] DELETE /identity/v3/projects/0cf29504a31e42e384c6318ae4be9e7a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:52.476038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8972cb-d014-4cb0-a6b3-69fda738f396 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:52.487046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8972cb-d014-4cb0-a6b3-69fda738f396 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:52.544791 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4797b4-d7ac-43d9-9513-eb3b278f4d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:52.545431 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4797b4-d7ac-43d9-9513-eb3b278f4d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:52.546493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4797b4-d7ac-43d9-9513-eb3b278f4d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:52.546642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4797b4-d7ac-43d9-9513-eb3b278f4d8a admin admin] PATH_INFO: `/v3/users/350d31ccdb02403c857a2e69a1c83bb5` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:52.547774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4797b4-d7ac-43d9-9513-eb3b278f4d8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=350d31ccdb02403c857a2e69a1c83bb5)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:52.549711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4797b4-d7ac-43d9-9513-eb3b278f4d8a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:52.598329 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2c4797b4-d7ac-43d9-9513-eb3b278f4d8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '350d31ccdb02403c857a2e69a1c83bb5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:52.605566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2c4797b4-d7ac-43d9-9513-eb3b278f4d8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '350d31ccdb02403c857a2e69a1c83bb5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:52.618823 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2c4797b4-d7ac-43d9-9513-eb3b278f4d8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '350d31ccdb02403c857a2e69a1c83bb5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:52.620595 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:52.628170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2c4797b4-d7ac-43d9-9513-eb3b278f4d8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '350d31ccdb02403c857a2e69a1c83bb5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:52.629809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2c4797b4-d7ac-43d9-9513-eb3b278f4d8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '350d31ccdb02403c857a2e69a1c83bb5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:52.636608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1326/2670] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:59:52 2022] DELETE /identity/v3/users/350d31ccdb02403c857a2e69a1c83bb5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:52.909467 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92caa8-4470-452a-9804-d9f4095624e5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:52.910979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-4b92caa8-4470-452a-9804-d9f4095624e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:52.925398 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92caa8-4470-452a-9804-d9f4095624e5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:52.984965 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa7de4-c0ce-4159-93e1-eeaae5c6a1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:52.985620 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa7de4-c0ce-4159-93e1-eeaae5c6a1d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:52.985975 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa7de4-c0ce-4159-93e1-eeaae5c6a1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:52.986115 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa7de4-c0ce-4159-93e1-eeaae5c6a1d9 admin admin] PATH_INFO: `/v3/projects/e5925a31c6e14e899aa90fb855bb02e2` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:52.987252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa7de4-c0ce-4159-93e1-eeaae5c6a1d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5925a31c6e14e899aa90fb855bb02e2)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:52.989260 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa7de4-c0ce-4159-93e1-eeaae5c6a1d9 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:53.040685 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-01aa7de4-c0ce-4159-93e1-eeaae5c6a1d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5925a31c6e14e899aa90fb855bb02e2', 'request_id': 'req-01aa7de4-c0ce-4159-93e1-eeaae5c6a1d9'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:53.046270 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-01aa7de4-c0ce-4159-93e1-eeaae5c6a1d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5925a31c6e14e899aa90fb855bb02e2', 'request_id': 'req-01aa7de4-c0ce-4159-93e1-eeaae5c6a1d9'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:53.053050 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-01aa7de4-c0ce-4159-93e1-eeaae5c6a1d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5925a31c6e14e899aa90fb855bb02e2', 'request_id': 'req-01aa7de4-c0ce-4159-93e1-eeaae5c6a1d9'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:53.069551 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-01aa7de4-c0ce-4159-93e1-eeaae5c6a1d9 admin admin] The token cache is being invalidate because project e5925a31c6e14e899aa90fb855bb02e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:59:53.069984 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-01aa7de4-c0ce-4159-93e1-eeaae5c6a1d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:53.070734 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-01aa7de4-c0ce-4159-93e1-eeaae5c6a1d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:53.073030 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1345/2671] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:59:52 2022] DELETE /identity/v3/projects/e5925a31c6e14e899aa90fb855bb02e2 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:53.078314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2d00e-7185-4d59-ac40-768a12c8e540 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:53.078615 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2d00e-7185-4d59-ac40-768a12c8e540 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:53.078895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2d00e-7185-4d59-ac40-768a12c8e540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:53.099473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-eef2d00e-7185-4d59-ac40-768a12c8e540 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:53.101382 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-eef2d00e-7185-4d59-ac40-768a12c8e540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:53.170961 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1327/2672] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:53.179198 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3218ce3-d010-404e-8e0a-a7e5224eaa65 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:53.179702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3218ce3-d010-404e-8e0a-a7e5224eaa65 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:53.180065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3218ce3-d010-404e-8e0a-a7e5224eaa65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:53.205517 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c3218ce3-d010-404e-8e0a-a7e5224eaa65 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:53.206831 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c3218ce3-d010-404e-8e0a-a7e5224eaa65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:53.215384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1346/2673] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:53.219785 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2d00e-7185-4d59-ac40-768a12c8e540 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:53.229747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2d00e-7185-4d59-ac40-768a12c8e540 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:53.234587 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daca68f-594d-4dfa-84fc-7b52bc2f282f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:53.235195 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daca68f-594d-4dfa-84fc-7b52bc2f282f admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:53.235479 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daca68f-594d-4dfa-84fc-7b52bc2f282f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:53.235849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daca68f-594d-4dfa-84fc-7b52bc2f282f admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:53.236506 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daca68f-594d-4dfa-84fc-7b52bc2f282f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 14:59:53.236809 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daca68f-594d-4dfa-84fc-7b52bc2f282f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:53.238462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daca68f-594d-4dfa-84fc-7b52bc2f282f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:53.251039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1328/2674] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 14:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 14:59:53.255480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3218ce3-d010-404e-8e0a-a7e5224eaa65 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:53.265759 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3218ce3-d010-404e-8e0a-a7e5224eaa65 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:53.270893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7fd59-619e-4160-9d37-f52387671a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:53.271656 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7fd59-619e-4160-9d37-f52387671a9a admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:53.271991 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7fd59-619e-4160-9d37-f52387671a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:53.272265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7fd59-619e-4160-9d37-f52387671a9a admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:53.272923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7fd59-619e-4160-9d37-f52387671a9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:53.274859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7fd59-619e-4160-9d37-f52387671a9a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:53.285301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:53.305318 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1347/2675] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 14:59:53 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:59:53.309829 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daca68f-594d-4dfa-84fc-7b52bc2f282f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:53.321924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daca68f-594d-4dfa-84fc-7b52bc2f282f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:53.394509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0f98e-cb30-4249-8fa9-525da60925f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:53.395160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0f98e-cb30-4249-8fa9-525da60925f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:53.395430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0f98e-cb30-4249-8fa9-525da60925f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:53.395701 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0f98e-cb30-4249-8fa9-525da60925f4 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:53.396375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0f98e-cb30-4249-8fa9-525da60925f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:53.398314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0f98e-cb30-4249-8fa9-525da60925f4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:53.420215 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1329/2676] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 14:59:53 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 14:59:53.424819 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7fd59-619e-4160-9d37-f52387671a9a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:53.437387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7fd59-619e-4160-9d37-f52387671a9a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:53.442696 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7732097-97f1-4dbf-a47f-de0a1f19f99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:53.444340 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7732097-97f1-4dbf-a47f-de0a1f19f99c admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:53.444748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7732097-97f1-4dbf-a47f-de0a1f19f99c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:53.445026 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7732097-97f1-4dbf-a47f-de0a1f19f99c admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:53.445697 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7732097-97f1-4dbf-a47f-de0a1f19f99c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:53.447394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7732097-97f1-4dbf-a47f-de0a1f19f99c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:53.458923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1348/2677] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:53.462525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0f98e-cb30-4249-8fa9-525da60925f4 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:53.473389 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0f98e-cb30-4249-8fa9-525da60925f4 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:53.479609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60fb2a4-c32d-4d17-b0c2-e618101ab0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:53.480283 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fb2a4-c32d-4d17-b0c2-e618101ab0da admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:53.480568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fb2a4-c32d-4d17-b0c2-e618101ab0da admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:53.480843 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60fb2a4-c32d-4d17-b0c2-e618101ab0da admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:53.481571 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60fb2a4-c32d-4d17-b0c2-e618101ab0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:53.483133 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60fb2a4-c32d-4d17-b0c2-e618101ab0da admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:53.495854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1330/2678] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 14:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 14:59:53.500654 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7732097-97f1-4dbf-a47f-de0a1f19f99c admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:53.510665 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7732097-97f1-4dbf-a47f-de0a1f19f99c admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:53.515249 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996c7d4-8b45-4968-ba38-ed5ceff5ed22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:53.516081 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996c7d4-8b45-4968-ba38-ed5ceff5ed22 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:53.516364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996c7d4-8b45-4968-ba38-ed5ceff5ed22 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:53.516637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1996c7d4-8b45-4968-ba38-ed5ceff5ed22 admin admin] PATH_INFO: `/v3/projects/64abe515cc83419c8b1f477aae9f15f3/users/328be27fa8764b0a81d3cfac3129baf1/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:53.530804 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996c7d4-8b45-4968-ba38-ed5ceff5ed22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64abe515cc83419c8b1f477aae9f15f3, user_id=328be27fa8764b0a81d3cfac3129baf1, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:53.533458 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1996c7d4-8b45-4968-ba38-ed5ceff5ed22 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:53.546646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1349/2679] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 14:59:53 2022] PUT /identity/v3/projects/64abe515cc83419c8b1f477aae9f15f3/users/328be27fa8764b0a81d3cfac3129baf1/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:53.556900 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82e5ab-9f24-48f7-845b-983dc45446bc None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:53.557444 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82e5ab-9f24-48f7-845b-983dc45446bc None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:53.557711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82e5ab-9f24-48f7-845b-983dc45446bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:53.585618 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9b82e5ab-9f24-48f7-845b-983dc45446bc None None] MFA Rules not processed for user `328be27fa8764b0a81d3cfac3129baf1`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:53.586847 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9b82e5ab-9f24-48f7-845b-983dc45446bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:53.634818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:53.657509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1331/2680] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 14:59:53.669868 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996c7d4-8b45-4968-ba38-ed5ceff5ed22 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:53.672265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1996c7d4-8b45-4968-ba38-ed5ceff5ed22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:53.732606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1996c7d4-8b45-4968-ba38-ed5ceff5ed22 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:53.737987 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a8866-d000-45fb-84f2-c375ddc15374 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caab7aaa72634b358d717768fbcb48d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:53.738606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a8866-d000-45fb-84f2-c375ddc15374 service glance] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:53.738898 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a8866-d000-45fb-84f2-c375ddc15374 service glance] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:53.739279 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a8866-d000-45fb-84f2-c375ddc15374 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:53.748390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a8866-d000-45fb-84f2-c375ddc15374 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:53.750399 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a8866-d000-45fb-84f2-c375ddc15374 service glance] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:53.759853 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1350/2681] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:55.530968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82e5ab-9f24-48f7-845b-983dc45446bc None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:55.532455 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9b82e5ab-9f24-48f7-845b-983dc45446bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:55.599150 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82e5ab-9f24-48f7-845b-983dc45446bc None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:55.604127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1f734-48a8-40f8-b50c-67f787c120dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:55.605176 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1f734-48a8-40f8-b50c-67f787c120dd service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:55.605666 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1f734-48a8-40f8-b50c-67f787c120dd service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:55.606154 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d1f734-48a8-40f8-b50c-67f787c120dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:55.609854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d5d1f734-48a8-40f8-b50c-67f787c120dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:55.687432 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1f734-48a8-40f8-b50c-67f787c120dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:55.688974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d1f734-48a8-40f8-b50c-67f787c120dd service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:55.699224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1332/2682] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 14:59:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 14:59:59.469235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a8866-d000-45fb-84f2-c375ddc15374 service glance] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:59.471253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fa3a8866-d000-45fb-84f2-c375ddc15374 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 14:59:59.489496 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a8866-d000-45fb-84f2-c375ddc15374 service glance] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:59.494041 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895421b-cccb-4de6-bad7-ec126ab82651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:59.494670 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895421b-cccb-4de6-bad7-ec126ab82651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:59.494951 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895421b-cccb-4de6-bad7-ec126ab82651 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:59.495256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895421b-cccb-4de6-bad7-ec126ab82651 admin admin] PATH_INFO: `/v3/users/73ef9d62f31d4135b6f079a26d791ca5` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:59.496430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895421b-cccb-4de6-bad7-ec126ab82651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73ef9d62f31d4135b6f079a26d791ca5)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:59.498323 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895421b-cccb-4de6-bad7-ec126ab82651 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:59.559426 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6895421b-cccb-4de6-bad7-ec126ab82651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73ef9d62f31d4135b6f079a26d791ca5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:59.567988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6895421b-cccb-4de6-bad7-ec126ab82651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73ef9d62f31d4135b6f079a26d791ca5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:59.581798 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6895421b-cccb-4de6-bad7-ec126ab82651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73ef9d62f31d4135b6f079a26d791ca5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:59.592523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6895421b-cccb-4de6-bad7-ec126ab82651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73ef9d62f31d4135b6f079a26d791ca5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:59.593291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-6895421b-cccb-4de6-bad7-ec126ab82651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73ef9d62f31d4135b6f079a26d791ca5'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:59.603226 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1351/2683] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 14:59:59 2022] DELETE /identity/v3/users/73ef9d62f31d4135b6f079a26d791ca5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:59.793637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1f734-48a8-40f8-b50c-67f787c120dd service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 14:59:59.804409 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d1f734-48a8-40f8-b50c-67f787c120dd service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 14:59:59.879012 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea7efd-d671-4edb-b403-22c8a842c369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 14:59:59.879639 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea7efd-d671-4edb-b403-22c8a842c369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:59.879969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea7efd-d671-4edb-b403-22c8a842c369 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:59.880254 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea7efd-d671-4edb-b403-22c8a842c369 admin admin] PATH_INFO: `/v3/projects/9a431be4c2f247e2955883eab6a5f555` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:59.881492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea7efd-d671-4edb-b403-22c8a842c369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a431be4c2f247e2955883eab6a5f555)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 14:59:59.883438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea7efd-d671-4edb-b403-22c8a842c369 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 14:59:59.931184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 14:59:59.940107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e5ea7efd-d671-4edb-b403-22c8a842c369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a431be4c2f247e2955883eab6a5f555', 'request_id': 'req-e5ea7efd-d671-4edb-b403-22c8a842c369'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:59.946920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e5ea7efd-d671-4edb-b403-22c8a842c369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a431be4c2f247e2955883eab6a5f555', 'request_id': 'req-e5ea7efd-d671-4edb-b403-22c8a842c369'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:59.955660 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e5ea7efd-d671-4edb-b403-22c8a842c369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a431be4c2f247e2955883eab6a5f555', 'request_id': 'req-e5ea7efd-d671-4edb-b403-22c8a842c369'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:59.966975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e5ea7efd-d671-4edb-b403-22c8a842c369 admin admin] The token cache is being invalidate because project 9a431be4c2f247e2955883eab6a5f555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 14:59:59.967265 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e5ea7efd-d671-4edb-b403-22c8a842c369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:59.968480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e5ea7efd-d671-4edb-b403-22c8a842c369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 14:59:59.970243 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1333/2684] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 14:59:59 2022] DELETE /identity/v3/projects/9a431be4c2f247e2955883eab6a5f555 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 14:59:59.975023 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2a1dc-63be-4c15-a8ff-025c265c7228 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 14:59:59.975310 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2a1dc-63be-4c15-a8ff-025c265c7228 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 14:59:59.975586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2a1dc-63be-4c15-a8ff-025c265c7228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 14:59:59.996225 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-c3d2a1dc-63be-4c15-a8ff-025c265c7228 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 14:59:59.997418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c3d2a1dc-63be-4c15-a8ff-025c265c7228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:00.024003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:00.064007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1352/2685] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 14:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:00.072481 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dc0f6-f713-4b54-a219-47c03557506f None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:00.072755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dc0f6-f713-4b54-a219-47c03557506f None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:00.073033 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dc0f6-f713-4b54-a219-47c03557506f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:00.094099 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-f56dc0f6-f713-4b54-a219-47c03557506f None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:00.095337 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f56dc0f6-f713-4b54-a219-47c03557506f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:00.104460 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1334/2686] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 15:00:00.109572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2a1dc-63be-4c15-a8ff-025c265c7228 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:00.120237 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2a1dc-63be-4c15-a8ff-025c265c7228 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:00.126170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fda7f-87bb-4e09-9199-69d2cdb9e859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:00.126947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fda7f-87bb-4e09-9199-69d2cdb9e859 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:00.127232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fda7f-87bb-4e09-9199-69d2cdb9e859 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:00.127504 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fda7f-87bb-4e09-9199-69d2cdb9e859 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:00.128187 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fda7f-87bb-4e09-9199-69d2cdb9e859 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:00:00.128521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fda7f-87bb-4e09-9199-69d2cdb9e859 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:00.130469 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fda7f-87bb-4e09-9199-69d2cdb9e859 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:00.143672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1353/2687] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 15:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:00:00.148168 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dc0f6-f713-4b54-a219-47c03557506f None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:00.160312 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dc0f6-f713-4b54-a219-47c03557506f None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:00.165738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf3b71-5cc5-46e5-a611-6aca8f4d7b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:00.166432 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf3b71-5cc5-46e5-a611-6aca8f4d7b5a admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:00.166714 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf3b71-5cc5-46e5-a611-6aca8f4d7b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:00.166986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf3b71-5cc5-46e5-a611-6aca8f4d7b5a admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:00.167641 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf3b71-5cc5-46e5-a611-6aca8f4d7b5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:00.169616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf3b71-5cc5-46e5-a611-6aca8f4d7b5a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:00.201076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1335/2688] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:00:00 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:00.205864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fda7f-87bb-4e09-9199-69d2cdb9e859 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:00.216870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fda7f-87bb-4e09-9199-69d2cdb9e859 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:00.274252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05016c93-b395-4c7f-8c9d-6a5152fb88dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:00.274879 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05016c93-b395-4c7f-8c9d-6a5152fb88dd admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:00.275155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05016c93-b395-4c7f-8c9d-6a5152fb88dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:00.275432 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05016c93-b395-4c7f-8c9d-6a5152fb88dd admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:00.276134 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05016c93-b395-4c7f-8c9d-6a5152fb88dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:00.278123 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05016c93-b395-4c7f-8c9d-6a5152fb88dd admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:00.300395 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1354/2689] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:00:00 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 15:00:00.304707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf3b71-5cc5-46e5-a611-6aca8f4d7b5a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:00.315582 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf3b71-5cc5-46e5-a611-6aca8f4d7b5a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:00.320415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2348d-87c0-477e-9f9e-56df2438af38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:00.321033 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2348d-87c0-477e-9f9e-56df2438af38 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:00.321318 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2348d-87c0-477e-9f9e-56df2438af38 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:00.321592 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2348d-87c0-477e-9f9e-56df2438af38 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:00.322263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2348d-87c0-477e-9f9e-56df2438af38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:00.323888 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2348d-87c0-477e-9f9e-56df2438af38 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:00.334692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1336/2690] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:00.338853 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05016c93-b395-4c7f-8c9d-6a5152fb88dd admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:00.349981 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05016c93-b395-4c7f-8c9d-6a5152fb88dd admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:00.354680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49434c6-dd5d-4214-96b5-a0ec742a4be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:00.355307 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49434c6-dd5d-4214-96b5-a0ec742a4be3 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:00.355597 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49434c6-dd5d-4214-96b5-a0ec742a4be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:00.355896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49434c6-dd5d-4214-96b5-a0ec742a4be3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:00.356562 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49434c6-dd5d-4214-96b5-a0ec742a4be3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:00.358180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49434c6-dd5d-4214-96b5-a0ec742a4be3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:00.369233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1355/2691] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 15:00:00.373496 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2348d-87c0-477e-9f9e-56df2438af38 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:00.383955 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2348d-87c0-477e-9f9e-56df2438af38 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:00.388854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5bb4b-59a5-408a-86ed-36ab92b50ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:00.390000 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5bb4b-59a5-408a-86ed-36ab92b50ce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:00.390277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5bb4b-59a5-408a-86ed-36ab92b50ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:00.390549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5bb4b-59a5-408a-86ed-36ab92b50ce9 admin admin] PATH_INFO: `/v3/projects/00bab4d0f0a0498aa4fd44a1ee4868e2/users/4125d04221d84aa2951e47b62f5554db/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:00.406842 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5bb4b-59a5-408a-86ed-36ab92b50ce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00bab4d0f0a0498aa4fd44a1ee4868e2, user_id=4125d04221d84aa2951e47b62f5554db, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:00.409555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5bb4b-59a5-408a-86ed-36ab92b50ce9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:00.422869 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1337/2692] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:00 2022] PUT /identity/v3/projects/00bab4d0f0a0498aa4fd44a1ee4868e2/users/4125d04221d84aa2951e47b62f5554db/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:00.434884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33566f0-0f87-465a-a98e-045d431aa54b None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:00.435234 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33566f0-0f87-465a-a98e-045d431aa54b None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:00.435438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33566f0-0f87-465a-a98e-045d431aa54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:00.464696 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a33566f0-0f87-465a-a98e-045d431aa54b None None] MFA Rules not processed for user `4125d04221d84aa2951e47b62f5554db`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:00.465827 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a33566f0-0f87-465a-a98e-045d431aa54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:00.535633 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1356/2693] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:00.540291 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5bb4b-59a5-408a-86ed-36ab92b50ce9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:00.607153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5bb4b-59a5-408a-86ed-36ab92b50ce9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:00.612105 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76cce7-220c-4917-8eb0-a0b6d369c82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:00.612723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76cce7-220c-4917-8eb0-a0b6d369c82a admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:00.612997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76cce7-220c-4917-8eb0-a0b6d369c82a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:00.613260 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f76cce7-220c-4917-8eb0-a0b6d369c82a admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:00.613887 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76cce7-220c-4917-8eb0-a0b6d369c82a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:00.615786 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f76cce7-220c-4917-8eb0-a0b6d369c82a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:00.649478 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1338/2694] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:00:00 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:00.653561 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33566f0-0f87-465a-a98e-045d431aa54b None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:00.664468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33566f0-0f87-465a-a98e-045d431aa54b None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:00.684867 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:00.724281 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20fd24-4089-465d-bba3-0fa51d516924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:00.724802 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20fd24-4089-465d-bba3-0fa51d516924 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:00.725091 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20fd24-4089-465d-bba3-0fa51d516924 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:00.725367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20fd24-4089-465d-bba3-0fa51d516924 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:00.726001 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20fd24-4089-465d-bba3-0fa51d516924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:00.727931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20fd24-4089-465d-bba3-0fa51d516924 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:00.752995 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1357/2695] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:00:00 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:00.757039 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76cce7-220c-4917-8eb0-a0b6d369c82a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:00.768024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f76cce7-220c-4917-8eb0-a0b6d369c82a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:00.773212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571f550-84fb-41f6-a69a-b86ad40e904c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:00.773865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571f550-84fb-41f6-a69a-b86ad40e904c admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:00.774203 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571f550-84fb-41f6-a69a-b86ad40e904c admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:00.774485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571f550-84fb-41f6-a69a-b86ad40e904c admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:00.775134 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7571f550-84fb-41f6-a69a-b86ad40e904c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:00.776783 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7571f550-84fb-41f6-a69a-b86ad40e904c admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:00.788100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1339/2696] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:00.792598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20fd24-4089-465d-bba3-0fa51d516924 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:00.802553 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20fd24-4089-465d-bba3-0fa51d516924 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:00.807440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa661b6e-8b2b-4ff1-a3e8-b88e85c7f48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:00.808135 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa661b6e-8b2b-4ff1-a3e8-b88e85c7f48f admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:00.808423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa661b6e-8b2b-4ff1-a3e8-b88e85c7f48f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:00.808696 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa661b6e-8b2b-4ff1-a3e8-b88e85c7f48f admin admin] PATH_INFO: `/v3/projects/30a2000d3f504dbf832002ff6c68b1e8/users/3f60177a19af4a1bbd91a7afdf435ee8/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:00.824500 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa661b6e-8b2b-4ff1-a3e8-b88e85c7f48f admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a2000d3f504dbf832002ff6c68b1e8, user_id=3f60177a19af4a1bbd91a7afdf435ee8, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:00.827769 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa661b6e-8b2b-4ff1-a3e8-b88e85c7f48f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:00.841387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1358/2697] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:00 2022] PUT /identity/v3/projects/30a2000d3f504dbf832002ff6c68b1e8/users/3f60177a19af4a1bbd91a7afdf435ee8/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:00.845611 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571f550-84fb-41f6-a69a-b86ad40e904c admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:00.855815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571f550-84fb-41f6-a69a-b86ad40e904c admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:00.891107 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:00.917668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf1dc0-1235-4f86-9b8c-3f4f942105e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:00.918284 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf1dc0-1235-4f86-9b8c-3f4f942105e4 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:00.918567 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf1dc0-1235-4f86-9b8c-3f4f942105e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:00.918842 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf1dc0-1235-4f86-9b8c-3f4f942105e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:00.919514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf1dc0-1235-4f86-9b8c-3f4f942105e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:00.921688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf1dc0-1235-4f86-9b8c-3f4f942105e4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:00.937570 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1340/2698] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:00.938952 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa661b6e-8b2b-4ff1-a3e8-b88e85c7f48f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:00.952849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa661b6e-8b2b-4ff1-a3e8-b88e85c7f48f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:00.958078 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48473000-a063-43c5-9f07-21b1c02c2b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:00.958748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48473000-a063-43c5-9f07-21b1c02c2b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:00.959023 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48473000-a063-43c5-9f07-21b1c02c2b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:00.959288 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48473000-a063-43c5-9f07-21b1c02c2b7f admin admin] PATH_INFO: `/v3/projects/30a2000d3f504dbf832002ff6c68b1e8/users/3f60177a19af4a1bbd91a7afdf435ee8/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:00.961205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48473000-a063-43c5-9f07-21b1c02c2b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a2000d3f504dbf832002ff6c68b1e8, user_id=3f60177a19af4a1bbd91a7afdf435ee8, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:00.963813 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48473000-a063-43c5-9f07-21b1c02c2b7f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:00.987729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1359/2699] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:00 2022] PUT /identity/v3/projects/30a2000d3f504dbf832002ff6c68b1e8/users/3f60177a19af4a1bbd91a7afdf435ee8/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:00.989452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecbc28-7b52-4833-a6f3-04a3a5b5181c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:00.989741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecbc28-7b52-4833-a6f3-04a3a5b5181c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:00.990176 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ecbc28-7b52-4833-a6f3-04a3a5b5181c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:01.019127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-91ecbc28-7b52-4833-a6f3-04a3a5b5181c None None] MFA Rules not processed for user `3f60177a19af4a1bbd91a7afdf435ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:01.020369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-91ecbc28-7b52-4833-a6f3-04a3a5b5181c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:01.108805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1341/2700] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 15:00:01.116353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48473000-a063-43c5-9f07-21b1c02c2b7f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:01.118045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-48473000-a063-43c5-9f07-21b1c02c2b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:01.178272 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48473000-a063-43c5-9f07-21b1c02c2b7f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:01.181974 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783ce1c-4034-4384-981f-896ab2422b32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:01.182848 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783ce1c-4034-4384-981f-896ab2422b32 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:01.183127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783ce1c-4034-4384-981f-896ab2422b32 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:01.183396 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783ce1c-4034-4384-981f-896ab2422b32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:01.193772 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f783ce1c-4034-4384-981f-896ab2422b32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:01.195331 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f783ce1c-4034-4384-981f-896ab2422b32 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:01.206501 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1360/2701] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:00:01.310065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ecbc28-7b52-4833-a6f3-04a3a5b5181c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:01.322851 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ecbc28-7b52-4833-a6f3-04a3a5b5181c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:01.329165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8998a8-129b-42fb-b0a3-2737b4639ccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:01.329851 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8998a8-129b-42fb-b0a3-2737b4639ccb service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:01.330052 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8998a8-129b-42fb-b0a3-2737b4639ccb service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:01.330317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8998a8-129b-42fb-b0a3-2737b4639ccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:01.399438 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8998a8-129b-42fb-b0a3-2737b4639ccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:01.400970 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8998a8-129b-42fb-b0a3-2737b4639ccb service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:01.410705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1342/2702] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 15:00:05.277761 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709844f-5701-42bb-9ae5-7c420d92d05a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:05.278218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709844f-5701-42bb-9ae5-7c420d92d05a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:05.278462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709844f-5701-42bb-9ae5-7c420d92d05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:05.299891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d709844f-5701-42bb-9ae5-7c420d92d05a None None] MFA Rules not processed for user `0edc61e9f06e47e69fad8ba5e2a7f201`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:05.301094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d709844f-5701-42bb-9ae5-7c420d92d05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:05.355412 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1361/2703] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:11.406171 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=74 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:13.622682 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8998a8-129b-42fb-b0a3-2737b4639ccb service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:13.624658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5d8998a8-129b-42fb-b0a3-2737b4639ccb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:13.693841 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8998a8-129b-42fb-b0a3-2737b4639ccb service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:13.698514 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51410d28-e00e-4944-9534-9e29d9198e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:13.699147 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51410d28-e00e-4944-9534-9e29d9198e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:13.699423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51410d28-e00e-4944-9534-9e29d9198e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:13.699690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51410d28-e00e-4944-9534-9e29d9198e3a admin admin] PATH_INFO: `/v3/users/328be27fa8764b0a81d3cfac3129baf1` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:13.701362 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51410d28-e00e-4944-9534-9e29d9198e3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=328be27fa8764b0a81d3cfac3129baf1)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:13.702788 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51410d28-e00e-4944-9534-9e29d9198e3a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:13.757940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-51410d28-e00e-4944-9534-9e29d9198e3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '328be27fa8764b0a81d3cfac3129baf1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:13.766957 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-51410d28-e00e-4944-9534-9e29d9198e3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '328be27fa8764b0a81d3cfac3129baf1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:13.779997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-51410d28-e00e-4944-9534-9e29d9198e3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '328be27fa8764b0a81d3cfac3129baf1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:13.787977 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-51410d28-e00e-4944-9534-9e29d9198e3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '328be27fa8764b0a81d3cfac3129baf1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:13.788621 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-51410d28-e00e-4944-9534-9e29d9198e3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '328be27fa8764b0a81d3cfac3129baf1'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:13.795865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1343/2704] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:00:13 2022] DELETE /identity/v3/users/328be27fa8764b0a81d3cfac3129baf1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:13.800478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e8449-d16c-48fd-8ca8-172e66cbd450 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:13.800988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e8449-d16c-48fd-8ca8-172e66cbd450 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:13.801090 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e8449-d16c-48fd-8ca8-172e66cbd450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:13.820648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ba3e8449-d16c-48fd-8ca8-172e66cbd450 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:13.821861 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ba3e8449-d16c-48fd-8ca8-172e66cbd450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:13.844850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:13.897678 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1362/2705] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:13.907892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51410d28-e00e-4944-9534-9e29d9198e3a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:13.909325 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-51410d28-e00e-4944-9534-9e29d9198e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:13.970978 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51410d28-e00e-4944-9534-9e29d9198e3a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:13.974713 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27faf8e2-dc87-438b-a116-9c0bd14b495d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:13.975322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27faf8e2-dc87-438b-a116-9c0bd14b495d service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:13.975599 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27faf8e2-dc87-438b-a116-9c0bd14b495d service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:13.975901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27faf8e2-dc87-438b-a116-9c0bd14b495d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:13.989021 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27faf8e2-dc87-438b-a116-9c0bd14b495d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:13.990518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27faf8e2-dc87-438b-a116-9c0bd14b495d service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:14.002069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1344/2706] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:00:14.321988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e8449-d16c-48fd-8ca8-172e66cbd450 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:14.323465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ba3e8449-d16c-48fd-8ca8-172e66cbd450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:14.336004 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e8449-d16c-48fd-8ca8-172e66cbd450 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:14.367975 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05fbec-ccf2-4787-96bd-e7b1a0ed848d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:14.370096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05fbec-ccf2-4787-96bd-e7b1a0ed848d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:14.371212 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05fbec-ccf2-4787-96bd-e7b1a0ed848d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:14.371870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05fbec-ccf2-4787-96bd-e7b1a0ed848d admin admin] PATH_INFO: `/v3/projects/64abe515cc83419c8b1f477aae9f15f3` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:14.374378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05fbec-ccf2-4787-96bd-e7b1a0ed848d admin admin] RBAC: Authorizing `identity:delete_project(project_id=64abe515cc83419c8b1f477aae9f15f3)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:14.378089 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05fbec-ccf2-4787-96bd-e7b1a0ed848d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:14.437199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-df05fbec-ccf2-4787-96bd-e7b1a0ed848d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64abe515cc83419c8b1f477aae9f15f3', 'request_id': 'req-df05fbec-ccf2-4787-96bd-e7b1a0ed848d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:14.442895 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-df05fbec-ccf2-4787-96bd-e7b1a0ed848d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64abe515cc83419c8b1f477aae9f15f3', 'request_id': 'req-df05fbec-ccf2-4787-96bd-e7b1a0ed848d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:14.449174 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-df05fbec-ccf2-4787-96bd-e7b1a0ed848d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64abe515cc83419c8b1f477aae9f15f3', 'request_id': 'req-df05fbec-ccf2-4787-96bd-e7b1a0ed848d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:14.463596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-df05fbec-ccf2-4787-96bd-e7b1a0ed848d admin admin] The token cache is being invalidate because project 64abe515cc83419c8b1f477aae9f15f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:00:14.464233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-df05fbec-ccf2-4787-96bd-e7b1a0ed848d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:14.464884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-df05fbec-ccf2-4787-96bd-e7b1a0ed848d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:14.466465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1363/2707] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:00:14 2022] DELETE /identity/v3/projects/64abe515cc83419c8b1f477aae9f15f3 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 15:00:14.472060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74abffe-b276-4b3e-b6be-0c93bf6e1230 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:14.472350 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74abffe-b276-4b3e-b6be-0c93bf6e1230 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:14.472620 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74abffe-b276-4b3e-b6be-0c93bf6e1230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:14.495057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b74abffe-b276-4b3e-b6be-0c93bf6e1230 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:14.496317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b74abffe-b276-4b3e-b6be-0c93bf6e1230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:14.517616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:14.563390 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1345/2708] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:14.572414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b895dd-fb76-4638-9b14-5587e903d9aa None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:14.572695 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b895dd-fb76-4638-9b14-5587e903d9aa None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:14.572970 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b895dd-fb76-4638-9b14-5587e903d9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:14.594945 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-f5b895dd-fb76-4638-9b14-5587e903d9aa None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:14.596201 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f5b895dd-fb76-4638-9b14-5587e903d9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:14.604286 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1364/2709] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:14.608815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74abffe-b276-4b3e-b6be-0c93bf6e1230 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:14.618501 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74abffe-b276-4b3e-b6be-0c93bf6e1230 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:14.623145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2ea95-ae08-4985-8a35-e2e6e7b9eb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:14.623735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2ea95-ae08-4985-8a35-e2e6e7b9eb9f admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:14.624044 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2ea95-ae08-4985-8a35-e2e6e7b9eb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:14.624321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2ea95-ae08-4985-8a35-e2e6e7b9eb9f admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:14.624959 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2ea95-ae08-4985-8a35-e2e6e7b9eb9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:00:14.625265 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2ea95-ae08-4985-8a35-e2e6e7b9eb9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:14.626871 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2ea95-ae08-4985-8a35-e2e6e7b9eb9f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:14.639244 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1346/2710] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 15:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:00:14.644112 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b895dd-fb76-4638-9b14-5587e903d9aa None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:14.658335 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b895dd-fb76-4638-9b14-5587e903d9aa None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:14.663918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e02b82-4679-441e-9cb6-f030d46ae0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:14.664548 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e02b82-4679-441e-9cb6-f030d46ae0b6 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:14.664827 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e02b82-4679-441e-9cb6-f030d46ae0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:14.665093 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e02b82-4679-441e-9cb6-f030d46ae0b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:14.665739 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e02b82-4679-441e-9cb6-f030d46ae0b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:14.667676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e02b82-4679-441e-9cb6-f030d46ae0b6 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:14.695788 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1365/2711] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:00:14 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 15:00:14.700231 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2ea95-ae08-4985-8a35-e2e6e7b9eb9f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:14.712523 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2ea95-ae08-4985-8a35-e2e6e7b9eb9f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:14.776146 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5190409-36cd-4872-b106-8b681606520d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:14.776865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5190409-36cd-4872-b106-8b681606520d admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:14.777217 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5190409-36cd-4872-b106-8b681606520d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:14.777650 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5190409-36cd-4872-b106-8b681606520d admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:14.778281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5190409-36cd-4872-b106-8b681606520d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:14.780903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5190409-36cd-4872-b106-8b681606520d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:14.802365 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1347/2712] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:00:14 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:14.806578 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e02b82-4679-441e-9cb6-f030d46ae0b6 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:14.817847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e02b82-4679-441e-9cb6-f030d46ae0b6 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:14.822466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea536b74-da79-4dec-8b70-79d1c9e97f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:14.823061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea536b74-da79-4dec-8b70-79d1c9e97f7e admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:14.823346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea536b74-da79-4dec-8b70-79d1c9e97f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:14.823620 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea536b74-da79-4dec-8b70-79d1c9e97f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:14.824312 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea536b74-da79-4dec-8b70-79d1c9e97f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:14.825882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea536b74-da79-4dec-8b70-79d1c9e97f7e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:14.836356 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1366/2713] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:14.840458 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5190409-36cd-4872-b106-8b681606520d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:14.850380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5190409-36cd-4872-b106-8b681606520d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:14.854978 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6db4d-3977-4c0c-9125-2440e4bb278f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:14.855569 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6db4d-3977-4c0c-9125-2440e4bb278f admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:14.855879 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6db4d-3977-4c0c-9125-2440e4bb278f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:14.856158 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6db4d-3977-4c0c-9125-2440e4bb278f admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:14.856786 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6db4d-3977-4c0c-9125-2440e4bb278f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:14.858386 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6db4d-3977-4c0c-9125-2440e4bb278f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:14.868849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1348/2714] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:14.873599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea536b74-da79-4dec-8b70-79d1c9e97f7e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:14.887677 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea536b74-da79-4dec-8b70-79d1c9e97f7e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:14.893211 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde15c25-dcd4-42e1-b7c8-3889e9bbc428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:14.893869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde15c25-dcd4-42e1-b7c8-3889e9bbc428 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:14.894155 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde15c25-dcd4-42e1-b7c8-3889e9bbc428 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:14.894452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde15c25-dcd4-42e1-b7c8-3889e9bbc428 admin admin] PATH_INFO: `/v3/projects/7f9b1ead149542d98759b87cb7b23b79/users/92425a2fce57430c9188bbbc8aa1152f/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:14.909295 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde15c25-dcd4-42e1-b7c8-3889e9bbc428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f9b1ead149542d98759b87cb7b23b79, user_id=92425a2fce57430c9188bbbc8aa1152f, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:14.912044 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde15c25-dcd4-42e1-b7c8-3889e9bbc428 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:14.926657 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1367/2715] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:14 2022] PUT /identity/v3/projects/7f9b1ead149542d98759b87cb7b23b79/users/92425a2fce57430c9188bbbc8aa1152f/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:14.938840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941118ed-ce59-41a0-8e9c-3442f3c6e093 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:14.939352 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941118ed-ce59-41a0-8e9c-3442f3c6e093 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:14.939815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941118ed-ce59-41a0-8e9c-3442f3c6e093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:14.968139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-941118ed-ce59-41a0-8e9c-3442f3c6e093 None None] MFA Rules not processed for user `92425a2fce57430c9188bbbc8aa1152f`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:14.969454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-941118ed-ce59-41a0-8e9c-3442f3c6e093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:15.034651 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1349/2716] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:15.046657 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde15c25-dcd4-42e1-b7c8-3889e9bbc428 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:15.048540 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-bde15c25-dcd4-42e1-b7c8-3889e9bbc428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:15.097359 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:15.110009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde15c25-dcd4-42e1-b7c8-3889e9bbc428 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:15.114006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a1a6f-a84a-41c8-8138-2ced63df094e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:15.114613 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a1a6f-a84a-41c8-8138-2ced63df094e service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:15.114891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a1a6f-a84a-41c8-8138-2ced63df094e service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:15.115159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a1a6f-a84a-41c8-8138-2ced63df094e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:15.124263 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a1a6f-a84a-41c8-8138-2ced63df094e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:15.125674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a1a6f-a84a-41c8-8138-2ced63df094e service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:15.134355 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1368/2717] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:00:17.717545 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941118ed-ce59-41a0-8e9c-3442f3c6e093 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:17.738115 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:17.789379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941118ed-ce59-41a0-8e9c-3442f3c6e093 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:17.794076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69ab68-1d69-4fa7-b005-d93647419bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:17.794708 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69ab68-1d69-4fa7-b005-d93647419bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:17.795552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69ab68-1d69-4fa7-b005-d93647419bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:17.795845 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69ab68-1d69-4fa7-b005-d93647419bb5 admin admin] PATH_INFO: `/v3/users/92425a2fce57430c9188bbbc8aa1152f` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:17.796972 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69ab68-1d69-4fa7-b005-d93647419bb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92425a2fce57430c9188bbbc8aa1152f)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:17.798918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69ab68-1d69-4fa7-b005-d93647419bb5 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:17.855445 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6b69ab68-1d69-4fa7-b005-d93647419bb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92425a2fce57430c9188bbbc8aa1152f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:17.867217 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6b69ab68-1d69-4fa7-b005-d93647419bb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92425a2fce57430c9188bbbc8aa1152f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:17.881847 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6b69ab68-1d69-4fa7-b005-d93647419bb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92425a2fce57430c9188bbbc8aa1152f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:17.890665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6b69ab68-1d69-4fa7-b005-d93647419bb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92425a2fce57430c9188bbbc8aa1152f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:17.891340 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6b69ab68-1d69-4fa7-b005-d93647419bb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92425a2fce57430c9188bbbc8aa1152f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:17.898702 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1350/2718] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:00:17 2022] DELETE /identity/v3/users/92425a2fce57430c9188bbbc8aa1152f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:17.903581 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0e8ae-4598-409f-927a-280ad9772512 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:17.904138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0e8ae-4598-409f-927a-280ad9772512 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:17.904138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d0e8ae-4598-409f-927a-280ad9772512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:17.925130 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b9d0e8ae-4598-409f-927a-280ad9772512 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:17.926470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b9d0e8ae-4598-409f-927a-280ad9772512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:17.988884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1369/2719] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:17.998600 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69ab68-1d69-4fa7-b005-d93647419bb5 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:18.000071 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6b69ab68-1d69-4fa7-b005-d93647419bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:18.061503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69ab68-1d69-4fa7-b005-d93647419bb5 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:18.065197 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15596217-3acf-4bea-8e20-322f89df6c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:18.065804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15596217-3acf-4bea-8e20-322f89df6c4b service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:18.066097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15596217-3acf-4bea-8e20-322f89df6c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:18.066357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15596217-3acf-4bea-8e20-322f89df6c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:18.075919 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15596217-3acf-4bea-8e20-322f89df6c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:18.077453 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15596217-3acf-4bea-8e20-322f89df6c4b service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:18.087668 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1351/2720] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 15:00:18.405548 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0e8ae-4598-409f-927a-280ad9772512 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:18.407046 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b9d0e8ae-4598-409f-927a-280ad9772512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:18.421739 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d0e8ae-4598-409f-927a-280ad9772512 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:18.442210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9a065-5141-4303-9629-04fa442a3607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:18.442895 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9a065-5141-4303-9629-04fa442a3607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:18.444357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9a065-5141-4303-9629-04fa442a3607 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:18.444658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9a065-5141-4303-9629-04fa442a3607 admin admin] PATH_INFO: `/v3/projects/7f9b1ead149542d98759b87cb7b23b79` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:18.447558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9a065-5141-4303-9629-04fa442a3607 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f9b1ead149542d98759b87cb7b23b79)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:18.449141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b9a065-5141-4303-9629-04fa442a3607 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:18.501366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c3b9a065-5141-4303-9629-04fa442a3607 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f9b1ead149542d98759b87cb7b23b79', 'request_id': 'req-c3b9a065-5141-4303-9629-04fa442a3607'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:18.507523 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c3b9a065-5141-4303-9629-04fa442a3607 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f9b1ead149542d98759b87cb7b23b79', 'request_id': 'req-c3b9a065-5141-4303-9629-04fa442a3607'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:18.514331 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c3b9a065-5141-4303-9629-04fa442a3607 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f9b1ead149542d98759b87cb7b23b79', 'request_id': 'req-c3b9a065-5141-4303-9629-04fa442a3607'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:18.527411 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c3b9a065-5141-4303-9629-04fa442a3607 admin admin] The token cache is being invalidate because project 7f9b1ead149542d98759b87cb7b23b79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:00:18.527705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c3b9a065-5141-4303-9629-04fa442a3607 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:18.528392 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-c3b9a065-5141-4303-9629-04fa442a3607 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:18.530616 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1370/2721] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:00:18 2022] DELETE /identity/v3/projects/7f9b1ead149542d98759b87cb7b23b79 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:18.535587 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1a505-65e3-4a1a-9141-871984ac338b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:18.536265 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1a505-65e3-4a1a-9141-871984ac338b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:18.536542 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1a505-65e3-4a1a-9141-871984ac338b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:18.560397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-e3d1a505-65e3-4a1a-9141-871984ac338b None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:18.562125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e3d1a505-65e3-4a1a-9141-871984ac338b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:18.595078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:18.621380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1352/2722] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:18.629568 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fefef4-d3c3-467b-a4dd-86209ed81602 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:18.629855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fefef4-d3c3-467b-a4dd-86209ed81602 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:18.630131 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fefef4-d3c3-467b-a4dd-86209ed81602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:18.651149 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-79fefef4-d3c3-467b-a4dd-86209ed81602 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:18.652387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-79fefef4-d3c3-467b-a4dd-86209ed81602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:18.660294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1371/2723] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:18.664670 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1a505-65e3-4a1a-9141-871984ac338b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:18.674380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1a505-65e3-4a1a-9141-871984ac338b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:18.679294 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3ab85-902c-4443-b765-18d49aea0bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:18.679794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3ab85-902c-4443-b765-18d49aea0bea admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:18.680114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3ab85-902c-4443-b765-18d49aea0bea admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:18.680392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3ab85-902c-4443-b765-18d49aea0bea admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:18.681030 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3ab85-902c-4443-b765-18d49aea0bea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:00:18.681334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3ab85-902c-4443-b765-18d49aea0bea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:18.683216 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3ab85-902c-4443-b765-18d49aea0bea admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:18.695365 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1353/2724] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 15:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:00:18.699905 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fefef4-d3c3-467b-a4dd-86209ed81602 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:18.710173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fefef4-d3c3-467b-a4dd-86209ed81602 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:18.714964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928f8c0-6efb-48a8-88ff-058529128f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:18.715558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928f8c0-6efb-48a8-88ff-058529128f66 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:18.715855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928f8c0-6efb-48a8-88ff-058529128f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:18.716138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928f8c0-6efb-48a8-88ff-058529128f66 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:18.716768 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928f8c0-6efb-48a8-88ff-058529128f66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:18.718715 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928f8c0-6efb-48a8-88ff-058529128f66 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:18.747345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1372/2725] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:00:18 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:18.752148 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3ab85-902c-4443-b765-18d49aea0bea admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:18.762114 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3ab85-902c-4443-b765-18d49aea0bea admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:18.820834 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b22abb-3f0c-4414-89bc-8218ebea73c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:18.821446 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b22abb-3f0c-4414-89bc-8218ebea73c0 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:18.821726 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b22abb-3f0c-4414-89bc-8218ebea73c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:18.821994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b22abb-3f0c-4414-89bc-8218ebea73c0 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:18.822622 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b22abb-3f0c-4414-89bc-8218ebea73c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:18.824579 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b22abb-3f0c-4414-89bc-8218ebea73c0 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:18.845448 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1354/2726] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:00:18 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:18.850425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928f8c0-6efb-48a8-88ff-058529128f66 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:18.860629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928f8c0-6efb-48a8-88ff-058529128f66 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:18.865632 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08488cf1-ba7a-48a3-8a32-0c78ed5c507e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:18.866263 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08488cf1-ba7a-48a3-8a32-0c78ed5c507e admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:18.866542 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08488cf1-ba7a-48a3-8a32-0c78ed5c507e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:18.866844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08488cf1-ba7a-48a3-8a32-0c78ed5c507e admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:18.867611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08488cf1-ba7a-48a3-8a32-0c78ed5c507e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:18.869364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08488cf1-ba7a-48a3-8a32-0c78ed5c507e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:18.880708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1373/2727] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:18.885379 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b22abb-3f0c-4414-89bc-8218ebea73c0 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:18.896761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b22abb-3f0c-4414-89bc-8218ebea73c0 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:18.901352 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb15e79-be05-4abb-94c9-3fe6ca89a0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:18.901944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb15e79-be05-4abb-94c9-3fe6ca89a0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:18.902224 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb15e79-be05-4abb-94c9-3fe6ca89a0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:18.902498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb15e79-be05-4abb-94c9-3fe6ca89a0b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:18.903125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb15e79-be05-4abb-94c9-3fe6ca89a0b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:18.904784 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb15e79-be05-4abb-94c9-3fe6ca89a0b9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:18.915410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1355/2728] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:18.919555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08488cf1-ba7a-48a3-8a32-0c78ed5c507e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:18.930202 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08488cf1-ba7a-48a3-8a32-0c78ed5c507e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:18.935078 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f499729-f972-4bea-8be2-05bc3d4d68c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:18.936386 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f499729-f972-4bea-8be2-05bc3d4d68c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:18.936657 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f499729-f972-4bea-8be2-05bc3d4d68c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:18.936921 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f499729-f972-4bea-8be2-05bc3d4d68c4 admin admin] PATH_INFO: `/v3/projects/b62ee0da1ed84a1288bc9e8ccf377cbe/users/4c32ce76e5c145048983199431c13f23/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:18.952255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f499729-f972-4bea-8be2-05bc3d4d68c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b62ee0da1ed84a1288bc9e8ccf377cbe, user_id=4c32ce76e5c145048983199431c13f23, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:18.954845 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f499729-f972-4bea-8be2-05bc3d4d68c4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:18.968001 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1374/2729] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:18 2022] PUT /identity/v3/projects/b62ee0da1ed84a1288bc9e8ccf377cbe/users/4c32ce76e5c145048983199431c13f23/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:18.979237 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b889540-6c3a-43d5-9d71-05749cac5e3c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:18.979555 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b889540-6c3a-43d5-9d71-05749cac5e3c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:18.979840 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b889540-6c3a-43d5-9d71-05749cac5e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:19.008374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1b889540-6c3a-43d5-9d71-05749cac5e3c None None] MFA Rules not processed for user `4c32ce76e5c145048983199431c13f23`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:19.009581 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1b889540-6c3a-43d5-9d71-05749cac5e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:19.083312 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1356/2730] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:19.094738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f499729-f972-4bea-8be2-05bc3d4d68c4 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:19.096561 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1f499729-f972-4bea-8be2-05bc3d4d68c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:19.106498 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:19.157771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f499729-f972-4bea-8be2-05bc3d4d68c4 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:19.161569 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49414385-a378-49e5-84cf-ad40b49cb5cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:19.162175 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49414385-a378-49e5-84cf-ad40b49cb5cb service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:19.162490 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49414385-a378-49e5-84cf-ad40b49cb5cb service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:19.162793 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49414385-a378-49e5-84cf-ad40b49cb5cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:19.172001 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49414385-a378-49e5-84cf-ad40b49cb5cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:19.172971 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49414385-a378-49e5-84cf-ad40b49cb5cb service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:19.181896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1375/2731] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:00:29.080364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:29.177720 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=23 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:30.850067 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c09dd-19ae-4b60-908b-2229ce9ebf6c None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:30.850568 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c09dd-19ae-4b60-908b-2229ce9ebf6c None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:30.850664 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c09dd-19ae-4b60-908b-2229ce9ebf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:30.897909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-619c09dd-19ae-4b60-908b-2229ce9ebf6c None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:30.899394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-619c09dd-19ae-4b60-908b-2229ce9ebf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:30.986981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1357/2732] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Sep 19 15:00:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:31.035719 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49414385-a378-49e5-84cf-ad40b49cb5cb service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:31.037199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-49414385-a378-49e5-84cf-ad40b49cb5cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:31.138319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49414385-a378-49e5-84cf-ad40b49cb5cb service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:31.144262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203190b-9185-4db2-9fa6-97ed552339c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:31.144865 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203190b-9185-4db2-9fa6-97ed552339c9 service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:31.145146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203190b-9185-4db2-9fa6-97ed552339c9 service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:31.145425 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203190b-9185-4db2-9fa6-97ed552339c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:31.153864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203190b-9185-4db2-9fa6-97ed552339c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:31.156011 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203190b-9185-4db2-9fa6-97ed552339c9 service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:31.164111 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1376/2733] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 15:00:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 15:00:38.373346 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c09dd-19ae-4b60-908b-2229ce9ebf6c None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:38.375061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-619c09dd-19ae-4b60-908b-2229ce9ebf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:38.480855 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c09dd-19ae-4b60-908b-2229ce9ebf6c None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:38.488058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a97e8d-8fec-4833-a75e-e3733c52c9fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '568189358b154b66a729fae2f0411d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:38.488866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a97e8d-8fec-4833-a75e-e3733c52c9fc service placement] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:38.489183 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a97e8d-8fec-4833-a75e-e3733c52c9fc service placement] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:38.489643 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a97e8d-8fec-4833-a75e-e3733c52c9fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:38.491580 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e0a97e8d-8fec-4833-a75e-e3733c52c9fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:38.505707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a97e8d-8fec-4833-a75e-e3733c52c9fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:38.508205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a97e8d-8fec-4833-a75e-e3733c52c9fc service placement] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:38.523918 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1358/2734] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:00:41.160801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=46 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:45.417306 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203190b-9185-4db2-9fa6-97ed552339c9 service swift] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:45.418905 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a203190b-9185-4db2-9fa6-97ed552339c9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:45.544372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203190b-9185-4db2-9fa6-97ed552339c9 service swift] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:45.549340 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371d967-7eaf-4d7b-9843-98c72f72e909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:45.549813 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371d967-7eaf-4d7b-9843-98c72f72e909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:45.550069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371d967-7eaf-4d7b-9843-98c72f72e909 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:45.550338 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371d967-7eaf-4d7b-9843-98c72f72e909 admin admin] PATH_INFO: `/v3/users/4125d04221d84aa2951e47b62f5554db` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:45.551506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371d967-7eaf-4d7b-9843-98c72f72e909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4125d04221d84aa2951e47b62f5554db)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:45.553937 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371d967-7eaf-4d7b-9843-98c72f72e909 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:45.644834 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a371d967-7eaf-4d7b-9843-98c72f72e909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4125d04221d84aa2951e47b62f5554db'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:45.655805 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a371d967-7eaf-4d7b-9843-98c72f72e909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4125d04221d84aa2951e47b62f5554db'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:45.675861 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a371d967-7eaf-4d7b-9843-98c72f72e909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4125d04221d84aa2951e47b62f5554db'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:45.686555 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a371d967-7eaf-4d7b-9843-98c72f72e909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4125d04221d84aa2951e47b62f5554db'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:45.688532 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a371d967-7eaf-4d7b-9843-98c72f72e909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4125d04221d84aa2951e47b62f5554db'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:45.696791 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1377/2735] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:00:45 2022] DELETE /identity/v3/users/4125d04221d84aa2951e47b62f5554db => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:45.702019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f56e02-ec83-431b-9114-7853e60c154d None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:45.702857 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f56e02-ec83-431b-9114-7853e60c154d None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:45.703295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f56e02-ec83-431b-9114-7853e60c154d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:45.741766 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a8f56e02-ec83-431b-9114-7853e60c154d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:45.744616 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a8f56e02-ec83-431b-9114-7853e60c154d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:45.772625 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=100 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:45.833556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1359/2736] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 15:00:45.846843 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371d967-7eaf-4d7b-9843-98c72f72e909 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:45.848330 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a371d967-7eaf-4d7b-9843-98c72f72e909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:45.921389 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:45.954500 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371d967-7eaf-4d7b-9843-98c72f72e909 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:45.959881 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3491a4-f3ec-4e10-a766-cfc18ed38e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:45.960738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3491a4-f3ec-4e10-a766-cfc18ed38e41 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:45.961020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3491a4-f3ec-4e10-a766-cfc18ed38e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:45.961298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3491a4-f3ec-4e10-a766-cfc18ed38e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:45.979602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3491a4-f3ec-4e10-a766-cfc18ed38e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:45.981170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3491a4-f3ec-4e10-a766-cfc18ed38e41 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:46.001929 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1378/2737] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:00:46.422944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f56e02-ec83-431b-9114-7853e60c154d None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:46.425279 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a8f56e02-ec83-431b-9114-7853e60c154d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:46.446222 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f56e02-ec83-431b-9114-7853e60c154d None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:46.465999 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f48eba-51f8-4f22-a36b-761195e3d770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:46.467008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f48eba-51f8-4f22-a36b-761195e3d770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:46.467532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f48eba-51f8-4f22-a36b-761195e3d770 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:46.468232 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f48eba-51f8-4f22-a36b-761195e3d770 admin admin] PATH_INFO: `/v3/projects/00bab4d0f0a0498aa4fd44a1ee4868e2` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:46.470026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f48eba-51f8-4f22-a36b-761195e3d770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00bab4d0f0a0498aa4fd44a1ee4868e2)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:46.472882 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f48eba-51f8-4f22-a36b-761195e3d770 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:46.532476 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c9f48eba-51f8-4f22-a36b-761195e3d770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00bab4d0f0a0498aa4fd44a1ee4868e2', 'request_id': 'req-c9f48eba-51f8-4f22-a36b-761195e3d770'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:46.541347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c9f48eba-51f8-4f22-a36b-761195e3d770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00bab4d0f0a0498aa4fd44a1ee4868e2', 'request_id': 'req-c9f48eba-51f8-4f22-a36b-761195e3d770'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:46.550990 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c9f48eba-51f8-4f22-a36b-761195e3d770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00bab4d0f0a0498aa4fd44a1ee4868e2', 'request_id': 'req-c9f48eba-51f8-4f22-a36b-761195e3d770'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:46.567896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c9f48eba-51f8-4f22-a36b-761195e3d770 admin admin] The token cache is being invalidate because project 00bab4d0f0a0498aa4fd44a1ee4868e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:00:46.567896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c9f48eba-51f8-4f22-a36b-761195e3d770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:46.567896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c9f48eba-51f8-4f22-a36b-761195e3d770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:46.567896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1360/2738] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:00:46 2022] DELETE /identity/v3/projects/00bab4d0f0a0498aa4fd44a1ee4868e2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:46.572197 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3491a4-f3ec-4e10-a766-cfc18ed38e41 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:46.583328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3491a4-f3ec-4e10-a766-cfc18ed38e41 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:46.661045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a78dc1-95e0-422c-afd5-22d3071a07cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:46.661712 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a78dc1-95e0-422c-afd5-22d3071a07cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:46.661998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a78dc1-95e0-422c-afd5-22d3071a07cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:46.662320 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a78dc1-95e0-422c-afd5-22d3071a07cf admin admin] PATH_INFO: `/v3/users/3f60177a19af4a1bbd91a7afdf435ee8` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:46.663517 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a78dc1-95e0-422c-afd5-22d3071a07cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f60177a19af4a1bbd91a7afdf435ee8)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:46.665551 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a78dc1-95e0-422c-afd5-22d3071a07cf admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:46.742218 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-59a78dc1-95e0-422c-afd5-22d3071a07cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f60177a19af4a1bbd91a7afdf435ee8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:46.755645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-59a78dc1-95e0-422c-afd5-22d3071a07cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f60177a19af4a1bbd91a7afdf435ee8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:46.776700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-59a78dc1-95e0-422c-afd5-22d3071a07cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f60177a19af4a1bbd91a7afdf435ee8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:46.786527 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-59a78dc1-95e0-422c-afd5-22d3071a07cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f60177a19af4a1bbd91a7afdf435ee8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:46.787154 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-59a78dc1-95e0-422c-afd5-22d3071a07cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f60177a19af4a1bbd91a7afdf435ee8'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:46.794445 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1379/2739] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:00:46 2022] DELETE /identity/v3/users/3f60177a19af4a1bbd91a7afdf435ee8 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:47.141194 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f48eba-51f8-4f22-a36b-761195e3d770 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:47.143135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c9f48eba-51f8-4f22-a36b-761195e3d770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:47.167132 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f48eba-51f8-4f22-a36b-761195e3d770 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:47.269981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0a1a2-360b-423a-94e8-c05313a77eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:47.271078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0a1a2-360b-423a-94e8-c05313a77eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:47.271584 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0a1a2-360b-423a-94e8-c05313a77eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:47.272108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0a1a2-360b-423a-94e8-c05313a77eb2 admin admin] PATH_INFO: `/v3/projects/30a2000d3f504dbf832002ff6c68b1e8` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:47.273620 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0a1a2-360b-423a-94e8-c05313a77eb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30a2000d3f504dbf832002ff6c68b1e8)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:47.275772 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0a1a2-360b-423a-94e8-c05313a77eb2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:47.333520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cdc0a1a2-360b-423a-94e8-c05313a77eb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30a2000d3f504dbf832002ff6c68b1e8', 'request_id': 'req-cdc0a1a2-360b-423a-94e8-c05313a77eb2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:47.335026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:47.344721 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cdc0a1a2-360b-423a-94e8-c05313a77eb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30a2000d3f504dbf832002ff6c68b1e8', 'request_id': 'req-cdc0a1a2-360b-423a-94e8-c05313a77eb2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:47.364509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cdc0a1a2-360b-423a-94e8-c05313a77eb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30a2000d3f504dbf832002ff6c68b1e8', 'request_id': 'req-cdc0a1a2-360b-423a-94e8-c05313a77eb2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:47.377527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cdc0a1a2-360b-423a-94e8-c05313a77eb2 admin admin] The token cache is being invalidate because project 30a2000d3f504dbf832002ff6c68b1e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:00:47.377866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cdc0a1a2-360b-423a-94e8-c05313a77eb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:47.378477 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cdc0a1a2-360b-423a-94e8-c05313a77eb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:47.385190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1361/2740] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:00:47 2022] DELETE /identity/v3/projects/30a2000d3f504dbf832002ff6c68b1e8 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:47.387100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dfbba-0823-4709-95c3-8924ebb46092 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:47.387914 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dfbba-0823-4709-95c3-8924ebb46092 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:47.388430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dfbba-0823-4709-95c3-8924ebb46092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:47.450777 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-cf1dfbba-0823-4709-95c3-8924ebb46092 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:47.452345 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-cf1dfbba-0823-4709-95c3-8924ebb46092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:47.483975 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:47.531863 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1380/2741] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:47.539563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8c973-2c3f-4ce4-9053-b2e178619477 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:47.540013 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8c973-2c3f-4ce4-9053-b2e178619477 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:47.540147 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8c973-2c3f-4ce4-9053-b2e178619477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:47.564129 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-3bc8c973-2c3f-4ce4-9053-b2e178619477 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:47.565885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-3bc8c973-2c3f-4ce4-9053-b2e178619477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:47.574593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1362/2742] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:47.579053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1dfbba-0823-4709-95c3-8924ebb46092 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:47.589600 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1dfbba-0823-4709-95c3-8924ebb46092 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:47.594522 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c41316-5f6f-4225-9c67-9d5ffa9b2d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:47.595208 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c41316-5f6f-4225-9c67-9d5ffa9b2d57 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:47.595706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c41316-5f6f-4225-9c67-9d5ffa9b2d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:47.596258 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c41316-5f6f-4225-9c67-9d5ffa9b2d57 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:47.596972 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c41316-5f6f-4225-9c67-9d5ffa9b2d57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:00:47.597391 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c41316-5f6f-4225-9c67-9d5ffa9b2d57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:47.599096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c41316-5f6f-4225-9c67-9d5ffa9b2d57 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:47.611733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1381/2743] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 15:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:00:47.616335 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8c973-2c3f-4ce4-9053-b2e178619477 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:47.626519 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8c973-2c3f-4ce4-9053-b2e178619477 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:47.631466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba8e0c-0cf6-4549-b845-6ad2bbae5db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:47.632190 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba8e0c-0cf6-4549-b845-6ad2bbae5db8 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:47.632474 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba8e0c-0cf6-4549-b845-6ad2bbae5db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:47.632740 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba8e0c-0cf6-4549-b845-6ad2bbae5db8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:47.633377 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba8e0c-0cf6-4549-b845-6ad2bbae5db8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:47.635284 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba8e0c-0cf6-4549-b845-6ad2bbae5db8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:47.665110 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1363/2744] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:00:47 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:47.671057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c41316-5f6f-4225-9c67-9d5ffa9b2d57 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:47.688244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c41316-5f6f-4225-9c67-9d5ffa9b2d57 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:47.767029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32766cf6-feb0-43ea-b076-0294e0b0f01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:47.767647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32766cf6-feb0-43ea-b076-0294e0b0f01a admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:47.767961 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32766cf6-feb0-43ea-b076-0294e0b0f01a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:47.768353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32766cf6-feb0-43ea-b076-0294e0b0f01a admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:47.769116 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32766cf6-feb0-43ea-b076-0294e0b0f01a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:47.771559 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32766cf6-feb0-43ea-b076-0294e0b0f01a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:47.806634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1382/2745] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:00:47 2022] POST /identity/v3/users => generated 293 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:47.811302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba8e0c-0cf6-4549-b845-6ad2bbae5db8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:47.830526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba8e0c-0cf6-4549-b845-6ad2bbae5db8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:47.835552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c90953-1efe-4787-ac94-7933cb27bf06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:47.836262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c90953-1efe-4787-ac94-7933cb27bf06 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:47.836556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c90953-1efe-4787-ac94-7933cb27bf06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:47.836836 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c90953-1efe-4787-ac94-7933cb27bf06 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:47.837455 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c90953-1efe-4787-ac94-7933cb27bf06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:47.839118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c90953-1efe-4787-ac94-7933cb27bf06 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:47.850650 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1364/2746] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:47.854766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32766cf6-feb0-43ea-b076-0294e0b0f01a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:47.866376 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32766cf6-feb0-43ea-b076-0294e0b0f01a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:47.871658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe923197-0806-45bf-829e-31129f63f5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:47.872390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe923197-0806-45bf-829e-31129f63f5d3 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:47.872736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe923197-0806-45bf-829e-31129f63f5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:47.873082 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe923197-0806-45bf-829e-31129f63f5d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:47.873895 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe923197-0806-45bf-829e-31129f63f5d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:47.875533 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe923197-0806-45bf-829e-31129f63f5d3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:47.890261 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1383/2747] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:47.894363 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c90953-1efe-4787-ac94-7933cb27bf06 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:47.905669 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c90953-1efe-4787-ac94-7933cb27bf06 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:47.910585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f961b-d797-42cf-82cf-bd501e6c9ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:47.911263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f961b-d797-42cf-82cf-bd501e6c9ec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:47.911579 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f961b-d797-42cf-82cf-bd501e6c9ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:47.911817 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f961b-d797-42cf-82cf-bd501e6c9ec1 admin admin] PATH_INFO: `/v3/projects/9850a23637244d19a3d7928a1be68a82/users/393834df77c64f72a1e19186684fb817/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:47.928325 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f961b-d797-42cf-82cf-bd501e6c9ec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9850a23637244d19a3d7928a1be68a82, user_id=393834df77c64f72a1e19186684fb817, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:47.930991 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f961b-d797-42cf-82cf-bd501e6c9ec1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:47.945973 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1365/2748] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:47 2022] PUT /identity/v3/projects/9850a23637244d19a3d7928a1be68a82/users/393834df77c64f72a1e19186684fb817/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:47.957923 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cbcc1-d3a5-4670-804f-ba9f70261c18 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:47.958216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cbcc1-d3a5-4670-804f-ba9f70261c18 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:47.958544 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cbcc1-d3a5-4670-804f-ba9f70261c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:47.992478 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-594cbcc1-d3a5-4670-804f-ba9f70261c18 None None] MFA Rules not processed for user `393834df77c64f72a1e19186684fb817`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:47.994053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-594cbcc1-d3a5-4670-804f-ba9f70261c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:48.072273 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1384/2749] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:48.073617 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f961b-d797-42cf-82cf-bd501e6c9ec1 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:48.158207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f961b-d797-42cf-82cf-bd501e6c9ec1 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:48.163294 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17591310-880a-4511-875e-caeb3e5695b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:48.164177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17591310-880a-4511-875e-caeb3e5695b6 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:48.164564 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17591310-880a-4511-875e-caeb3e5695b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:48.164965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17591310-880a-4511-875e-caeb3e5695b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:48.165944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17591310-880a-4511-875e-caeb3e5695b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:48.168298 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17591310-880a-4511-875e-caeb3e5695b6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:48.205169 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1366/2750] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:00:48 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:48.210869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cbcc1-d3a5-4670-804f-ba9f70261c18 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:48.225710 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594cbcc1-d3a5-4670-804f-ba9f70261c18 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:48.258722 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:48.324079 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c796cb-8c2a-4964-afd2-809ab343f4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:48.324079 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c796cb-8c2a-4964-afd2-809ab343f4fe admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:48.324079 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c796cb-8c2a-4964-afd2-809ab343f4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:48.324079 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c796cb-8c2a-4964-afd2-809ab343f4fe admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:48.325294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c796cb-8c2a-4964-afd2-809ab343f4fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:48.327647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c796cb-8c2a-4964-afd2-809ab343f4fe admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:48.356236 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1385/2751] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:00:48 2022] POST /identity/v3/users => generated 299 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:48.358077 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17591310-880a-4511-875e-caeb3e5695b6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:48.370923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17591310-880a-4511-875e-caeb3e5695b6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:48.377905 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90eadb-bfdd-4e5c-916a-785d60258d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:48.378556 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90eadb-bfdd-4e5c-916a-785d60258d25 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:48.378888 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90eadb-bfdd-4e5c-916a-785d60258d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:48.379159 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90eadb-bfdd-4e5c-916a-785d60258d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:48.379803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90eadb-bfdd-4e5c-916a-785d60258d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:48.382205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90eadb-bfdd-4e5c-916a-785d60258d25 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:48.393722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1367/2752] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:48.398258 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c796cb-8c2a-4964-afd2-809ab343f4fe admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:48.408763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c796cb-8c2a-4964-afd2-809ab343f4fe admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:48.413636 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43033d-996d-4d30-884c-f6eb8aa209a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:48.414299 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43033d-996d-4d30-884c-f6eb8aa209a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:48.414585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43033d-996d-4d30-884c-f6eb8aa209a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:48.414862 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf43033d-996d-4d30-884c-f6eb8aa209a4 admin admin] PATH_INFO: `/v3/projects/f16dcdd5479e45939d5c0ec4999106e2/users/648980d7a1e44e4e850a5975a1f0db0e/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:48.433243 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43033d-996d-4d30-884c-f6eb8aa209a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f16dcdd5479e45939d5c0ec4999106e2, user_id=648980d7a1e44e4e850a5975a1f0db0e, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:48.436286 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf43033d-996d-4d30-884c-f6eb8aa209a4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:48.451818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1386/2753] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:48 2022] PUT /identity/v3/projects/f16dcdd5479e45939d5c0ec4999106e2/users/648980d7a1e44e4e850a5975a1f0db0e/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:48.456552 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90eadb-bfdd-4e5c-916a-785d60258d25 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:48.469661 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90eadb-bfdd-4e5c-916a-785d60258d25 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:48.536704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:48.555372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b2175-9b0e-4640-a65c-6e6f806b389e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:48.556205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b2175-9b0e-4640-a65c-6e6f806b389e admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:48.556431 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b2175-9b0e-4640-a65c-6e6f806b389e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:48.556866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b2175-9b0e-4640-a65c-6e6f806b389e admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:48.558468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b2175-9b0e-4640-a65c-6e6f806b389e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:48.559868 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9b2175-9b0e-4640-a65c-6e6f806b389e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:48.575949 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1368/2754] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 15:00:48.578139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43033d-996d-4d30-884c-f6eb8aa209a4 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:48.588974 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf43033d-996d-4d30-884c-f6eb8aa209a4 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:48.594010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acee4d-4ae4-4c8f-89cb-f962e9c65931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:48.594705 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acee4d-4ae4-4c8f-89cb-f962e9c65931 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:48.594968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acee4d-4ae4-4c8f-89cb-f962e9c65931 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:48.595245 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acee4d-4ae4-4c8f-89cb-f962e9c65931 admin admin] PATH_INFO: `/v3/projects/f16dcdd5479e45939d5c0ec4999106e2/users/648980d7a1e44e4e850a5975a1f0db0e/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:48.597290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70acee4d-4ae4-4c8f-89cb-f962e9c65931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f16dcdd5479e45939d5c0ec4999106e2, user_id=648980d7a1e44e4e850a5975a1f0db0e, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:48.599963 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70acee4d-4ae4-4c8f-89cb-f962e9c65931 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:48.611582 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1387/2755] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:48 2022] PUT /identity/v3/projects/f16dcdd5479e45939d5c0ec4999106e2/users/648980d7a1e44e4e850a5975a1f0db0e/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:48.621945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99b1e2-671d-4c9f-840c-1a8de482aa00 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:48.622241 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99b1e2-671d-4c9f-840c-1a8de482aa00 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:48.622521 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed99b1e2-671d-4c9f-840c-1a8de482aa00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:48.655000 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ed99b1e2-671d-4c9f-840c-1a8de482aa00 None None] MFA Rules not processed for user `648980d7a1e44e4e850a5975a1f0db0e`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:48.656258 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ed99b1e2-671d-4c9f-840c-1a8de482aa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:48.734158 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1369/2756] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:48.749367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acee4d-4ae4-4c8f-89cb-f962e9c65931 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:48.751055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-70acee4d-4ae4-4c8f-89cb-f962e9c65931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:48.848173 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acee4d-4ae4-4c8f-89cb-f962e9c65931 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:48.852477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f783f8-d02e-4374-ae8a-15a1c03675d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:48.853280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f783f8-d02e-4374-ae8a-15a1c03675d2 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:48.853814 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f783f8-d02e-4374-ae8a-15a1c03675d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:48.854212 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f783f8-d02e-4374-ae8a-15a1c03675d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:48.864870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f783f8-d02e-4374-ae8a-15a1c03675d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:48.867240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f783f8-d02e-4374-ae8a-15a1c03675d2 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:48.880915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1388/2757] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 15:00:48.916008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99b1e2-671d-4c9f-840c-1a8de482aa00 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:48.987356 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed99b1e2-671d-4c9f-840c-1a8de482aa00 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:48.992462 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311275ea-a61b-4720-b5e0-cf53b35d6ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:48.993186 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311275ea-a61b-4720-b5e0-cf53b35d6ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:48.993535 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311275ea-a61b-4720-b5e0-cf53b35d6ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:48.993908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311275ea-a61b-4720-b5e0-cf53b35d6ec7 admin admin] PATH_INFO: `/v3/users/393834df77c64f72a1e19186684fb817` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:48.995262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311275ea-a61b-4720-b5e0-cf53b35d6ec7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=393834df77c64f72a1e19186684fb817)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:48.997292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311275ea-a61b-4720-b5e0-cf53b35d6ec7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:49.055531 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-311275ea-a61b-4720-b5e0-cf53b35d6ec7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '393834df77c64f72a1e19186684fb817'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:49.069147 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-311275ea-a61b-4720-b5e0-cf53b35d6ec7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '393834df77c64f72a1e19186684fb817'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:49.093561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:49.095125 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-311275ea-a61b-4720-b5e0-cf53b35d6ec7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '393834df77c64f72a1e19186684fb817'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:49.107540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-311275ea-a61b-4720-b5e0-cf53b35d6ec7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '393834df77c64f72a1e19186684fb817'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:49.108517 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-311275ea-a61b-4720-b5e0-cf53b35d6ec7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '393834df77c64f72a1e19186684fb817'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:49.117435 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1370/2758] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:00:48 2022] DELETE /identity/v3/users/393834df77c64f72a1e19186684fb817 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 15:00:49.122438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a50409-63c0-4098-a088-6cfec7d31753 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:49.122725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a50409-63c0-4098-a088-6cfec7d31753 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:49.123008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a50409-63c0-4098-a088-6cfec7d31753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:49.148314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-22a50409-63c0-4098-a088-6cfec7d31753 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:49.149719 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-22a50409-63c0-4098-a088-6cfec7d31753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:49.213870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:49.224873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1389/2759] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:49.238371 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311275ea-a61b-4720-b5e0-cf53b35d6ec7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:49.239935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-311275ea-a61b-4720-b5e0-cf53b35d6ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:49.303138 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311275ea-a61b-4720-b5e0-cf53b35d6ec7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:49.306987 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82829de8-653f-44ee-922d-f54ffff0f63a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:49.308711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82829de8-653f-44ee-922d-f54ffff0f63a service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:49.308881 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82829de8-653f-44ee-922d-f54ffff0f63a service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:49.309123 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82829de8-653f-44ee-922d-f54ffff0f63a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:49.319785 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82829de8-653f-44ee-922d-f54ffff0f63a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:49.321282 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82829de8-653f-44ee-922d-f54ffff0f63a service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:49.334920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1371/2760] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:00:49.688381 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a50409-63c0-4098-a088-6cfec7d31753 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:49.690188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-22a50409-63c0-4098-a088-6cfec7d31753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:49.706763 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a50409-63c0-4098-a088-6cfec7d31753 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:49.713561 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:49.714666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:49.715042 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:49.716433 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79 admin admin] PATH_INFO: `/v3/projects/9850a23637244d19a3d7928a1be68a82` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:49.718437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9850a23637244d19a3d7928a1be68a82)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:49.721034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:49.792595 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9850a23637244d19a3d7928a1be68a82', 'request_id': 'req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:49.800907 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9850a23637244d19a3d7928a1be68a82', 'request_id': 'req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:49.808179 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9850a23637244d19a3d7928a1be68a82', 'request_id': 'req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:49.826950 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79 admin admin] The token cache is being invalidate because project 9850a23637244d19a3d7928a1be68a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:00:49.827347 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:49.828031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:49.829596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1390/2761] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:00:49 2022] DELETE /identity/v3/projects/9850a23637244d19a3d7928a1be68a82 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 15:00:49.834870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82829de8-653f-44ee-922d-f54ffff0f63a service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:49.847367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82829de8-653f-44ee-922d-f54ffff0f63a service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:49.917003 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cf78e-7d36-4fa3-9a79-3ded4ccd2519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:49.918113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cf78e-7d36-4fa3-9a79-3ded4ccd2519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:49.918258 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cf78e-7d36-4fa3-9a79-3ded4ccd2519 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:49.918494 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cf78e-7d36-4fa3-9a79-3ded4ccd2519 admin admin] PATH_INFO: `/v3/users/648980d7a1e44e4e850a5975a1f0db0e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:49.919675 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0cf78e-7d36-4fa3-9a79-3ded4ccd2519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=648980d7a1e44e4e850a5975a1f0db0e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:49.922774 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0cf78e-7d36-4fa3-9a79-3ded4ccd2519 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:49.984206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc0cf78e-7d36-4fa3-9a79-3ded4ccd2519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '648980d7a1e44e4e850a5975a1f0db0e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:49.992805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc0cf78e-7d36-4fa3-9a79-3ded4ccd2519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '648980d7a1e44e4e850a5975a1f0db0e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:50.008572 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc0cf78e-7d36-4fa3-9a79-3ded4ccd2519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '648980d7a1e44e4e850a5975a1f0db0e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:50.017612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc0cf78e-7d36-4fa3-9a79-3ded4ccd2519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '648980d7a1e44e4e850a5975a1f0db0e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:50.018731 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-bc0cf78e-7d36-4fa3-9a79-3ded4ccd2519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '648980d7a1e44e4e850a5975a1f0db0e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:50.027236 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1372/2762] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:00:49 2022] DELETE /identity/v3/users/648980d7a1e44e4e850a5975a1f0db0e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:50.356742 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:50.358595 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:50.388690 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9d63ac-2ef7-4054-9cbe-2e51811dcb79 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:50.480399 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f78e15-d7bd-41e3-bf01-cffaa660ab67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:50.481055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f78e15-d7bd-41e3-bf01-cffaa660ab67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:50.481328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f78e15-d7bd-41e3-bf01-cffaa660ab67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:50.481676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f78e15-d7bd-41e3-bf01-cffaa660ab67 admin admin] PATH_INFO: `/v3/projects/f16dcdd5479e45939d5c0ec4999106e2` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:50.483328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f78e15-d7bd-41e3-bf01-cffaa660ab67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f16dcdd5479e45939d5c0ec4999106e2)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:50.485287 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f78e15-d7bd-41e3-bf01-cffaa660ab67 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:50.542869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f9f78e15-d7bd-41e3-bf01-cffaa660ab67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f16dcdd5479e45939d5c0ec4999106e2', 'request_id': 'req-f9f78e15-d7bd-41e3-bf01-cffaa660ab67'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:50.551645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f9f78e15-d7bd-41e3-bf01-cffaa660ab67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f16dcdd5479e45939d5c0ec4999106e2', 'request_id': 'req-f9f78e15-d7bd-41e3-bf01-cffaa660ab67'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:50.562045 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f9f78e15-d7bd-41e3-bf01-cffaa660ab67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f16dcdd5479e45939d5c0ec4999106e2', 'request_id': 'req-f9f78e15-d7bd-41e3-bf01-cffaa660ab67'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:50.577878 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f9f78e15-d7bd-41e3-bf01-cffaa660ab67 admin admin] The token cache is being invalidate because project f16dcdd5479e45939d5c0ec4999106e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:00:50.578163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f9f78e15-d7bd-41e3-bf01-cffaa660ab67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:50.578911 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-f9f78e15-d7bd-41e3-bf01-cffaa660ab67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:50.580325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1391/2763] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:00:50 2022] DELETE /identity/v3/projects/f16dcdd5479e45939d5c0ec4999106e2 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:50.586439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc4993-c6cf-4631-8a9d-9e92c9a75897 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:50.586939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc4993-c6cf-4631-8a9d-9e92c9a75897 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:50.589050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc4993-c6cf-4631-8a9d-9e92c9a75897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:50.613854 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-51cc4993-c6cf-4631-8a9d-9e92c9a75897 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:50.615642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-51cc4993-c6cf-4631-8a9d-9e92c9a75897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:50.623204 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:50.710792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1373/2764] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:50.724163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a707f-e6cb-4d0a-b18d-19367f7628ae None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:50.724766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a707f-e6cb-4d0a-b18d-19367f7628ae None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:50.724766 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a707f-e6cb-4d0a-b18d-19367f7628ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:50.746640 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e11a707f-e6cb-4d0a-b18d-19367f7628ae None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:50.747936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e11a707f-e6cb-4d0a-b18d-19367f7628ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:50.756960 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1392/2765] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:50.761822 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc4993-c6cf-4631-8a9d-9e92c9a75897 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:50.773439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc4993-c6cf-4631-8a9d-9e92c9a75897 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:50.778718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc548f8-30f5-42d5-8983-541c27a8352f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:50.780007 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc548f8-30f5-42d5-8983-541c27a8352f admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:50.780292 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc548f8-30f5-42d5-8983-541c27a8352f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:50.780742 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc548f8-30f5-42d5-8983-541c27a8352f admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:50.781413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc548f8-30f5-42d5-8983-541c27a8352f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:00:50.781716 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc548f8-30f5-42d5-8983-541c27a8352f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:50.783385 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc548f8-30f5-42d5-8983-541c27a8352f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:50.798426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1374/2766] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 15:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:00:50.803855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a707f-e6cb-4d0a-b18d-19367f7628ae None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:50.816252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a707f-e6cb-4d0a-b18d-19367f7628ae None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:50.822187 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac287db-4feb-4f81-a612-c59a4b5694ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:50.822900 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac287db-4feb-4f81-a612-c59a4b5694ef admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:50.823303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac287db-4feb-4f81-a612-c59a4b5694ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:50.823572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac287db-4feb-4f81-a612-c59a4b5694ef admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:50.824468 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac287db-4feb-4f81-a612-c59a4b5694ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:50.826814 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac287db-4feb-4f81-a612-c59a4b5694ef admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:50.838876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:50.902397 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1393/2767] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:00:50 2022] POST /identity/v3/projects => generated 393 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:50.906623 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc548f8-30f5-42d5-8983-541c27a8352f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:50.917375 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc548f8-30f5-42d5-8983-541c27a8352f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:50.980293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8a6d8-41de-4872-b07c-fe36074ab226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:50.981113 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8a6d8-41de-4872-b07c-fe36074ab226 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:50.981407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8a6d8-41de-4872-b07c-fe36074ab226 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:50.981675 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8a6d8-41de-4872-b07c-fe36074ab226 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:50.982323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8a6d8-41de-4872-b07c-fe36074ab226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:50.984870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8a6d8-41de-4872-b07c-fe36074ab226 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:51.017896 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1375/2768] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:00:50 2022] POST /identity/v3/users => generated 297 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:51.023002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac287db-4feb-4f81-a612-c59a4b5694ef admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:51.037770 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac287db-4feb-4f81-a612-c59a4b5694ef admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:51.043325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e25d7f-3934-4e75-a975-009a3fca1a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:51.044089 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e25d7f-3934-4e75-a975-009a3fca1a69 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:51.044484 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e25d7f-3934-4e75-a975-009a3fca1a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:51.044936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e25d7f-3934-4e75-a975-009a3fca1a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:51.045739 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e25d7f-3934-4e75-a975-009a3fca1a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:51.047462 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e25d7f-3934-4e75-a975-009a3fca1a69 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:51.059407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1394/2769] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 15:00:51.064975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8a6d8-41de-4872-b07c-fe36074ab226 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:51.078767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8a6d8-41de-4872-b07c-fe36074ab226 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:51.085253 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ddb18-2737-4c0d-b912-e8b4ad606219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:51.086166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ddb18-2737-4c0d-b912-e8b4ad606219 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:51.086507 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ddb18-2737-4c0d-b912-e8b4ad606219 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:51.086786 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ddb18-2737-4c0d-b912-e8b4ad606219 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:51.087433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ddb18-2737-4c0d-b912-e8b4ad606219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:51.094910 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ddb18-2737-4c0d-b912-e8b4ad606219 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:51.111710 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e25d7f-3934-4e75-a975-009a3fca1a69 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:51.112465 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1376/2770] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:51.124203 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e25d7f-3934-4e75-a975-009a3fca1a69 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:51.130590 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da748687-44d8-431f-8886-b7f4b5b4bab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:51.131602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da748687-44d8-431f-8886-b7f4b5b4bab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:51.132204 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da748687-44d8-431f-8886-b7f4b5b4bab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:51.132776 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da748687-44d8-431f-8886-b7f4b5b4bab0 admin admin] PATH_INFO: `/v3/projects/45f510bcf4554f60a3c3b5fc1a521cfc/users/8fd15d8665ab48148f6ae0c092cf5e6c/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:51.157384 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da748687-44d8-431f-8886-b7f4b5b4bab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f510bcf4554f60a3c3b5fc1a521cfc, user_id=8fd15d8665ab48148f6ae0c092cf5e6c, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:51.161298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da748687-44d8-431f-8886-b7f4b5b4bab0 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:51.180556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1395/2771] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:51 2022] PUT /identity/v3/projects/45f510bcf4554f60a3c3b5fc1a521cfc/users/8fd15d8665ab48148f6ae0c092cf5e6c/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 15:00:51.194947 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af5664-754f-4df9-8d25-734aebe8b192 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:51.195225 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af5664-754f-4df9-8d25-734aebe8b192 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:51.195688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af5664-754f-4df9-8d25-734aebe8b192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:51.229937 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-d6af5664-754f-4df9-8d25-734aebe8b192 None None] MFA Rules not processed for user `8fd15d8665ab48148f6ae0c092cf5e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:51.231545 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d6af5664-754f-4df9-8d25-734aebe8b192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:51.320734 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1377/2772] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:51.325692 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da748687-44d8-431f-8886-b7f4b5b4bab0 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:51.405170 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da748687-44d8-431f-8886-b7f4b5b4bab0 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:51.410759 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71ad7b-2419-4a30-9056-0649e5954506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:51.411650 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71ad7b-2419-4a30-9056-0649e5954506 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:51.412237 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71ad7b-2419-4a30-9056-0649e5954506 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:51.412689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71ad7b-2419-4a30-9056-0649e5954506 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:51.413776 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71ad7b-2419-4a30-9056-0649e5954506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:51.416060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71ad7b-2419-4a30-9056-0649e5954506 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:51.446778 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1396/2773] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:00:51 2022] POST /identity/v3/projects => generated 391 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:51.451263 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af5664-754f-4df9-8d25-734aebe8b192 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:51.464878 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af5664-754f-4df9-8d25-734aebe8b192 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:51.469805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:51.539651 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097042f-135c-4a68-ba91-34f6a863de06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:51.540415 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097042f-135c-4a68-ba91-34f6a863de06 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:51.540781 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097042f-135c-4a68-ba91-34f6a863de06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:51.541126 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8097042f-135c-4a68-ba91-34f6a863de06 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:51.541837 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097042f-135c-4a68-ba91-34f6a863de06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:51.543852 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8097042f-135c-4a68-ba91-34f6a863de06 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:51.576020 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1378/2774] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:00:51 2022] POST /identity/v3/users => generated 302 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:51.580002 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71ad7b-2419-4a30-9056-0649e5954506 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:51.595127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71ad7b-2419-4a30-9056-0649e5954506 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:51.602882 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1a6b4-201b-4436-aa6a-591947d271a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:51.603623 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1a6b4-201b-4436-aa6a-591947d271a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:51.604030 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1a6b4-201b-4436-aa6a-591947d271a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:51.604386 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1a6b4-201b-4436-aa6a-591947d271a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:51.605124 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1a6b4-201b-4436-aa6a-591947d271a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:51.606827 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d1a6b4-201b-4436-aa6a-591947d271a8 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:51.621151 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1397/2775] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:51.625493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097042f-135c-4a68-ba91-34f6a863de06 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:51.637923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8097042f-135c-4a68-ba91-34f6a863de06 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:51.646935 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f8105-60fb-45c9-9645-8f9566fc2ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:51.647672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f8105-60fb-45c9-9645-8f9566fc2ff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:51.647994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f8105-60fb-45c9-9645-8f9566fc2ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:51.648273 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f8105-60fb-45c9-9645-8f9566fc2ff8 admin admin] PATH_INFO: `/v3/projects/ed03e9a09f894d65b763130b332de271/users/7c460da32e314bf79fbbd56e79feea7a/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:51.667177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f8105-60fb-45c9-9645-8f9566fc2ff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed03e9a09f894d65b763130b332de271, user_id=7c460da32e314bf79fbbd56e79feea7a, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:51.668539 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f8105-60fb-45c9-9645-8f9566fc2ff8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:51.685761 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1379/2776] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:51 2022] PUT /identity/v3/projects/ed03e9a09f894d65b763130b332de271/users/7c460da32e314bf79fbbd56e79feea7a/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:51.690048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1a6b4-201b-4436-aa6a-591947d271a8 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:51.701819 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d1a6b4-201b-4436-aa6a-591947d271a8 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:51.775736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedbf765-29c8-49af-ad9b-f32cbfcb0dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:51.776414 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbf765-29c8-49af-ad9b-f32cbfcb0dcc admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:51.776673 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbf765-29c8-49af-ad9b-f32cbfcb0dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:51.776946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedbf765-29c8-49af-ad9b-f32cbfcb0dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:51.777579 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbf765-29c8-49af-ad9b-f32cbfcb0dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:51.779206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedbf765-29c8-49af-ad9b-f32cbfcb0dcc admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:51.790421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1398/2777] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:51.796305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f8105-60fb-45c9-9645-8f9566fc2ff8 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:51.806601 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f8105-60fb-45c9-9645-8f9566fc2ff8 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:51.811565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298833e7-db0e-4f9a-885c-cbc31bcac338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:51.812245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298833e7-db0e-4f9a-885c-cbc31bcac338 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:51.812526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298833e7-db0e-4f9a-885c-cbc31bcac338 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:51.812803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298833e7-db0e-4f9a-885c-cbc31bcac338 admin admin] PATH_INFO: `/v3/projects/ed03e9a09f894d65b763130b332de271/users/7c460da32e314bf79fbbd56e79feea7a/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:51.814783 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298833e7-db0e-4f9a-885c-cbc31bcac338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed03e9a09f894d65b763130b332de271, user_id=7c460da32e314bf79fbbd56e79feea7a, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:51.817553 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298833e7-db0e-4f9a-885c-cbc31bcac338 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:51.830286 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1380/2778] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:51 2022] PUT /identity/v3/projects/ed03e9a09f894d65b763130b332de271/users/7c460da32e314bf79fbbd56e79feea7a/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:51.844034 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ab2da-5f10-4b56-b64e-013ad0715a24 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:51.844328 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ab2da-5f10-4b56-b64e-013ad0715a24 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:51.844608 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ab2da-5f10-4b56-b64e-013ad0715a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:51.878700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1b8ab2da-5f10-4b56-b64e-013ad0715a24 None None] MFA Rules not processed for user `7c460da32e314bf79fbbd56e79feea7a`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:51.880222 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1b8ab2da-5f10-4b56-b64e-013ad0715a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:51.890057 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:51.953651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1399/2779] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:51 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:51.964384 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298833e7-db0e-4f9a-885c-cbc31bcac338 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:51.966098 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-298833e7-db0e-4f9a-885c-cbc31bcac338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:52.024593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298833e7-db0e-4f9a-885c-cbc31bcac338 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:52.028262 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40eea80-5c32-4237-9ec9-95126fa21f00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:52.028866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40eea80-5c32-4237-9ec9-95126fa21f00 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:52.029146 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40eea80-5c32-4237-9ec9-95126fa21f00 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:52.030026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40eea80-5c32-4237-9ec9-95126fa21f00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:52.039091 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40eea80-5c32-4237-9ec9-95126fa21f00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:52.040558 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40eea80-5c32-4237-9ec9-95126fa21f00 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:52.050628 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1381/2780] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:51 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:00:52.099248 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ab2da-5f10-4b56-b64e-013ad0715a24 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:52.100681 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1b8ab2da-5f10-4b56-b64e-013ad0715a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:52.113168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ab2da-5f10-4b56-b64e-013ad0715a24 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:52.116849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdce063-0b38-48d4-9181-6690136559a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:52.117452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdce063-0b38-48d4-9181-6690136559a9 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:52.117735 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdce063-0b38-48d4-9181-6690136559a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:52.118008 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdce063-0b38-48d4-9181-6690136559a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:52.176792 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdce063-0b38-48d4-9181-6690136559a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:52.178219 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdce063-0b38-48d4-9181-6690136559a9 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:52.189097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1400/2781] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:00:53.010493 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eab317-53fb-47c5-b28a-6d6f46406446 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:53.011046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eab317-53fb-47c5-b28a-6d6f46406446 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:53.011046 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eab317-53fb-47c5-b28a-6d6f46406446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:53.030468 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-97eab317-53fb-47c5-b28a-6d6f46406446 None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:53.031663 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-97eab317-53fb-47c5-b28a-6d6f46406446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:53.088448 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1382/2782] 10.222.0.60 () {58 vars in 967 bytes} [Mon Sep 19 15:00:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:53.102183 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdce063-0b38-48d4-9181-6690136559a9 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:53.103592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1cdce063-0b38-48d4-9181-6690136559a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:53.176465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdce063-0b38-48d4-9181-6690136559a9 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:53.182069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e103838-0a88-4401-8692-487f35c869c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:53.182956 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e103838-0a88-4401-8692-487f35c869c9 service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:53.183240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e103838-0a88-4401-8692-487f35c869c9 service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:53.183513 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e103838-0a88-4401-8692-487f35c869c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:53.192840 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e103838-0a88-4401-8692-487f35c869c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:53.194464 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e103838-0a88-4401-8692-487f35c869c9 service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:53.203141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1401/2783] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 15:00:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 15:00:55.166828 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eab317-53fb-47c5-b28a-6d6f46406446 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:55.168450 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-97eab317-53fb-47c5-b28a-6d6f46406446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:55.179707 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:55.239747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eab317-53fb-47c5-b28a-6d6f46406446 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:55.244679 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed10e94-3c2a-4bf4-8428-d5592c135eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:55.245317 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed10e94-3c2a-4bf4-8428-d5592c135eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:55.245594 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed10e94-3c2a-4bf4-8428-d5592c135eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:55.245866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed10e94-3c2a-4bf4-8428-d5592c135eb1 admin admin] PATH_INFO: `/v3/users/0edc61e9f06e47e69fad8ba5e2a7f201` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:55.247052 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed10e94-3c2a-4bf4-8428-d5592c135eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0edc61e9f06e47e69fad8ba5e2a7f201)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:55.249656 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed10e94-3c2a-4bf4-8428-d5592c135eb1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:55.298170 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2ed10e94-3c2a-4bf4-8428-d5592c135eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0edc61e9f06e47e69fad8ba5e2a7f201'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:55.306097 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2ed10e94-3c2a-4bf4-8428-d5592c135eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0edc61e9f06e47e69fad8ba5e2a7f201'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:55.318869 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2ed10e94-3c2a-4bf4-8428-d5592c135eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0edc61e9f06e47e69fad8ba5e2a7f201'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:55.327049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2ed10e94-3c2a-4bf4-8428-d5592c135eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0edc61e9f06e47e69fad8ba5e2a7f201'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:55.327803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2ed10e94-3c2a-4bf4-8428-d5592c135eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0edc61e9f06e47e69fad8ba5e2a7f201'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:55.334502 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1383/2784] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:00:55 2022] DELETE /identity/v3/users/0edc61e9f06e47e69fad8ba5e2a7f201 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:55.524443 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e103838-0a88-4401-8692-487f35c869c9 service swift] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:55.540240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e103838-0a88-4401-8692-487f35c869c9 service swift] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:55.575323 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed10e94-3c2a-4bf4-8428-d5592c135eb1 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:55.576824 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2ed10e94-3c2a-4bf4-8428-d5592c135eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:55.582336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=100 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:55.612048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8a839-e95e-427c-a6b5-76bdf68a82e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:55.612601 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8a839-e95e-427c-a6b5-76bdf68a82e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:55.612884 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8a839-e95e-427c-a6b5-76bdf68a82e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:55.613159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8a839-e95e-427c-a6b5-76bdf68a82e3 admin admin] PATH_INFO: `/v3/projects/4502d9a0ff3a414f855bf0beb35ffd35` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:55.614316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8a839-e95e-427c-a6b5-76bdf68a82e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4502d9a0ff3a414f855bf0beb35ffd35)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:55.616281 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8a839-e95e-427c-a6b5-76bdf68a82e3 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:55.654191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed10e94-3c2a-4bf4-8428-d5592c135eb1 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:55.668006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78c8a839-e95e-427c-a6b5-76bdf68a82e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4502d9a0ff3a414f855bf0beb35ffd35', 'request_id': 'req-78c8a839-e95e-427c-a6b5-76bdf68a82e3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:55.674371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78c8a839-e95e-427c-a6b5-76bdf68a82e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4502d9a0ff3a414f855bf0beb35ffd35', 'request_id': 'req-78c8a839-e95e-427c-a6b5-76bdf68a82e3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:55.681192 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78c8a839-e95e-427c-a6b5-76bdf68a82e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4502d9a0ff3a414f855bf0beb35ffd35', 'request_id': 'req-78c8a839-e95e-427c-a6b5-76bdf68a82e3'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:55.687597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:55.696821 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78c8a839-e95e-427c-a6b5-76bdf68a82e3 admin admin] The token cache is being invalidate because project 4502d9a0ff3a414f855bf0beb35ffd35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:00:55.697146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78c8a839-e95e-427c-a6b5-76bdf68a82e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:55.697801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-78c8a839-e95e-427c-a6b5-76bdf68a82e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:55.699296 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1402/2785] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:00:55 2022] DELETE /identity/v3/projects/4502d9a0ff3a414f855bf0beb35ffd35 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:55.704103 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b7a42-c489-4c39-9d00-837688e1ba6e None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:55.704470 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b7a42-c489-4c39-9d00-837688e1ba6e None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:55.704793 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b7a42-c489-4c39-9d00-837688e1ba6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:55.719700 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2ba4ec-da19-4184-b86a-6f82bc56e420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:55.720397 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ba4ec-da19-4184-b86a-6f82bc56e420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:55.720681 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ba4ec-da19-4184-b86a-6f82bc56e420 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:55.721344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2ba4ec-da19-4184-b86a-6f82bc56e420 admin admin] PATH_INFO: `/v3/users/8fd15d8665ab48148f6ae0c092cf5e6c` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:55.723004 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ba4ec-da19-4184-b86a-6f82bc56e420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fd15d8665ab48148f6ae0c092cf5e6c)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:55.725424 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2ba4ec-da19-4184-b86a-6f82bc56e420 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:55.733848 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-3b0b7a42-c489-4c39-9d00-837688e1ba6e None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:55.735353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3b0b7a42-c489-4c39-9d00-837688e1ba6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:55.746038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1403/2786] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 15:00:55.755018 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97270cb7-b0b5-4800-962e-4c94d651e651 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:55.755301 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97270cb7-b0b5-4800-962e-4c94d651e651 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:55.755592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97270cb7-b0b5-4800-962e-4c94d651e651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:55.788040 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fe2ba4ec-da19-4184-b86a-6f82bc56e420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fd15d8665ab48148f6ae0c092cf5e6c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:55.789985 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-97270cb7-b0b5-4800-962e-4c94d651e651 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:55.791259 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-97270cb7-b0b5-4800-962e-4c94d651e651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:55.795750 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fe2ba4ec-da19-4184-b86a-6f82bc56e420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fd15d8665ab48148f6ae0c092cf5e6c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:55.799893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1404/2787] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 15:00:55.804253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97270cb7-b0b5-4800-962e-4c94d651e651 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:55.808654 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fe2ba4ec-da19-4184-b86a-6f82bc56e420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fd15d8665ab48148f6ae0c092cf5e6c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:55.817932 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97270cb7-b0b5-4800-962e-4c94d651e651 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:55.818157 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fe2ba4ec-da19-4184-b86a-6f82bc56e420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fd15d8665ab48148f6ae0c092cf5e6c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:55.818770 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-fe2ba4ec-da19-4184-b86a-6f82bc56e420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fd15d8665ab48148f6ae0c092cf5e6c'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:55.822982 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69b1e1-2e15-49c7-93c8-0a60750d9c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:55.823614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69b1e1-2e15-49c7-93c8-0a60750d9c94 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:55.823920 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69b1e1-2e15-49c7-93c8-0a60750d9c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:55.824193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c69b1e1-2e15-49c7-93c8-0a60750d9c94 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:55.824836 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69b1e1-2e15-49c7-93c8-0a60750d9c94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:00:55.825136 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69b1e1-2e15-49c7-93c8-0a60750d9c94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:55.826777 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c69b1e1-2e15-49c7-93c8-0a60750d9c94 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:55.827903 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1384/2788] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:00:55 2022] DELETE /identity/v3/users/8fd15d8665ab48148f6ae0c092cf5e6c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:55.830302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a5732-bce7-4682-968b-8a21765e96e0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:55.830590 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a5732-bce7-4682-968b-8a21765e96e0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:55.830866 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a5732-bce7-4682-968b-8a21765e96e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:55.839784 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1405/2789] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 15:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 19 15:00:55.844825 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69b1e1-2e15-49c7-93c8-0a60750d9c94 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:55.846979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-2c69b1e1-2e15-49c7-93c8-0a60750d9c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:55.852036 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-232a5732-bce7-4682-968b-8a21765e96e0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:55.853225 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-232a5732-bce7-4682-968b-8a21765e96e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:55.860466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c69b1e1-2e15-49c7-93c8-0a60750d9c94 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:55.921019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1385/2790] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 15:00:55.930060 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e37ab-b0fc-4773-be01-e974262cc377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:55.930689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e37ab-b0fc-4773-be01-e974262cc377 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:55.930966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e37ab-b0fc-4773-be01-e974262cc377 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:55.931138 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a5732-bce7-4682-968b-8a21765e96e0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:55.931253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e37ab-b0fc-4773-be01-e974262cc377 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:55.931881 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e37ab-b0fc-4773-be01-e974262cc377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:55.933024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-232a5732-bce7-4682-968b-8a21765e96e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:55.934098 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e37ab-b0fc-4773-be01-e974262cc377 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:55.967658 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1406/2791] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:00:55 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 15:00:55.971981 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e37ab-b0fc-4773-be01-e974262cc377 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:55.994349 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a5732-bce7-4682-968b-8a21765e96e0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:55.998545 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:55.999623 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:56.038250 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e37ab-b0fc-4773-be01-e974262cc377 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:56.045325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f07067-6a0a-445d-96a5-772d1dfb0160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:56.045325 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f07067-6a0a-445d-96a5-772d1dfb0160 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:56.045606 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f07067-6a0a-445d-96a5-772d1dfb0160 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:56.046539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f07067-6a0a-445d-96a5-772d1dfb0160 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:56.046539 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f07067-6a0a-445d-96a5-772d1dfb0160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:56.049382 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f07067-6a0a-445d-96a5-772d1dfb0160 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:56.058527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8385bc9-4303-4732-b392-dd6f1b717e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:56.059182 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8385bc9-4303-4732-b392-dd6f1b717e57 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:56.059492 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8385bc9-4303-4732-b392-dd6f1b717e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:56.059764 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8385bc9-4303-4732-b392-dd6f1b717e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:56.070380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8385bc9-4303-4732-b392-dd6f1b717e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:56.072226 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1407/2792] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:00:55 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 15:00:56.072393 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8385bc9-4303-4732-b392-dd6f1b717e57 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:56.076946 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f07067-6a0a-445d-96a5-772d1dfb0160 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:56.084578 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1386/2793] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 15:00:56.089172 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f07067-6a0a-445d-96a5-772d1dfb0160 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:56.094265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949564e-4168-48e3-bff9-871bb5faaac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:56.094881 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949564e-4168-48e3-bff9-871bb5faaac7 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:56.095165 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949564e-4168-48e3-bff9-871bb5faaac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:56.095520 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949564e-4168-48e3-bff9-871bb5faaac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:56.096108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949564e-4168-48e3-bff9-871bb5faaac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:56.097871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9949564e-4168-48e3-bff9-871bb5faaac7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:56.111144 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1408/2794] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 15:00:56.114981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8385bc9-4303-4732-b392-dd6f1b717e57 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:56.125895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8385bc9-4303-4732-b392-dd6f1b717e57 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:56.131284 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02d0816-7b3a-4d77-892f-d1453d1dcff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:56.132234 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d0816-7b3a-4d77-892f-d1453d1dcff1 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:56.132516 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d0816-7b3a-4d77-892f-d1453d1dcff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:56.132789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02d0816-7b3a-4d77-892f-d1453d1dcff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:56.133634 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d0816-7b3a-4d77-892f-d1453d1dcff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:56.135287 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02d0816-7b3a-4d77-892f-d1453d1dcff1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:56.151615 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1387/2795] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:56.153255 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949564e-4168-48e3-bff9-871bb5faaac7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:56.165599 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9949564e-4168-48e3-bff9-871bb5faaac7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:56.171421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6d781-cf27-47e8-a338-e6f735a620ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:56.172097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6d781-cf27-47e8-a338-e6f735a620ba admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:56.172377 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6d781-cf27-47e8-a338-e6f735a620ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:56.172651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6d781-cf27-47e8-a338-e6f735a620ba admin admin] PATH_INFO: `/v3/projects/eb4bd6bb69c04dcf8475595904675e20/users/18bd94a24883495b9fbe066ddc2a953d/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:56.189378 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6d781-cf27-47e8-a338-e6f735a620ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb4bd6bb69c04dcf8475595904675e20, user_id=18bd94a24883495b9fbe066ddc2a953d, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:56.193232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd6d781-cf27-47e8-a338-e6f735a620ba admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:56.219475 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1409/2796] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:56 2022] PUT /identity/v3/projects/eb4bd6bb69c04dcf8475595904675e20/users/18bd94a24883495b9fbe066ddc2a953d/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:56.221307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d850dde-5f06-4a39-bea4-024ae8cc6db2 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:56.221921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d850dde-5f06-4a39-bea4-024ae8cc6db2 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:56.221921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d850dde-5f06-4a39-bea4-024ae8cc6db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:56.262553 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-7d850dde-5f06-4a39-bea4-024ae8cc6db2 None None] MFA Rules not processed for user `18bd94a24883495b9fbe066ddc2a953d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:56.264049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7d850dde-5f06-4a39-bea4-024ae8cc6db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:56.332508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1388/2797] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:56.348609 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6d781-cf27-47e8-a338-e6f735a620ba admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:56.350508 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-fdd6d781-cf27-47e8-a338-e6f735a620ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:56.394374 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d850dde-5f06-4a39-bea4-024ae8cc6db2 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:56.395988 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-7d850dde-5f06-4a39-bea4-024ae8cc6db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:56.418891 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6d781-cf27-47e8-a338-e6f735a620ba admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:56.423564 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474f004-3208-443b-bb1e-49815e5ae601 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caab7aaa72634b358d717768fbcb48d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:56.424235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474f004-3208-443b-bb1e-49815e5ae601 service glance] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:56.424516 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474f004-3208-443b-bb1e-49815e5ae601 service glance] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:56.425040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474f004-3208-443b-bb1e-49815e5ae601 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:56.437168 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d474f004-3208-443b-bb1e-49815e5ae601 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:56.438611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d474f004-3208-443b-bb1e-49815e5ae601 service glance] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:56.448046 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1410/2798] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:00:56.479749 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d850dde-5f06-4a39-bea4-024ae8cc6db2 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:56.485373 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:56.486026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:56.486316 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:56.486590 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4 admin admin] PATH_INFO: `/v3/projects/45f510bcf4554f60a3c3b5fc1a521cfc` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:56.487746 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45f510bcf4554f60a3c3b5fc1a521cfc)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:56.489730 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:56.530255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:56.544750 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45f510bcf4554f60a3c3b5fc1a521cfc', 'request_id': 'req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:56.551430 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45f510bcf4554f60a3c3b5fc1a521cfc', 'request_id': 'req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:56.562778 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45f510bcf4554f60a3c3b5fc1a521cfc', 'request_id': 'req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:56.574983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4 admin admin] The token cache is being invalidate because project 45f510bcf4554f60a3c3b5fc1a521cfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:00:56.575302 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:56.576525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:56.578037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1389/2799] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:00:56 2022] DELETE /identity/v3/projects/45f510bcf4554f60a3c3b5fc1a521cfc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 15:00:56.582206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474f004-3208-443b-bb1e-49815e5ae601 service glance] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:56.594021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474f004-3208-443b-bb1e-49815e5ae601 service glance] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:56.659068 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254cf7e4-8bd9-43fe-9703-d27e1e6b4acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:56.659708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cf7e4-8bd9-43fe-9703-d27e1e6b4acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:56.660038 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cf7e4-8bd9-43fe-9703-d27e1e6b4acb admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:56.660319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254cf7e4-8bd9-43fe-9703-d27e1e6b4acb admin admin] PATH_INFO: `/v3/users/7c460da32e314bf79fbbd56e79feea7a` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:56.661461 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254cf7e4-8bd9-43fe-9703-d27e1e6b4acb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c460da32e314bf79fbbd56e79feea7a)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:56.663366 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254cf7e4-8bd9-43fe-9703-d27e1e6b4acb admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:56.688628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:56.719674 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-254cf7e4-8bd9-43fe-9703-d27e1e6b4acb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c460da32e314bf79fbbd56e79feea7a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:56.728560 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-254cf7e4-8bd9-43fe-9703-d27e1e6b4acb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c460da32e314bf79fbbd56e79feea7a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:56.743652 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-254cf7e4-8bd9-43fe-9703-d27e1e6b4acb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c460da32e314bf79fbbd56e79feea7a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:56.753331 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-254cf7e4-8bd9-43fe-9703-d27e1e6b4acb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c460da32e314bf79fbbd56e79feea7a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:56.754145 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-254cf7e4-8bd9-43fe-9703-d27e1e6b4acb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c460da32e314bf79fbbd56e79feea7a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:56.761053 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1411/2800] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:00:56 2022] DELETE /identity/v3/users/7c460da32e314bf79fbbd56e79feea7a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 15:00:57.079596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:57.081068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:57.096753 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bd65a-ede9-4af9-ba37-fbc350a727e4 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:57.167015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51843284-41a0-4a79-b76a-a872919db5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:57.167908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51843284-41a0-4a79-b76a-a872919db5e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:57.168307 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51843284-41a0-4a79-b76a-a872919db5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:57.168582 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51843284-41a0-4a79-b76a-a872919db5e8 admin admin] PATH_INFO: `/v3/projects/ed03e9a09f894d65b763130b332de271` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:57.170406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51843284-41a0-4a79-b76a-a872919db5e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed03e9a09f894d65b763130b332de271)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:57.172226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51843284-41a0-4a79-b76a-a872919db5e8 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:57.221245 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-51843284-41a0-4a79-b76a-a872919db5e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed03e9a09f894d65b763130b332de271', 'request_id': 'req-51843284-41a0-4a79-b76a-a872919db5e8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:57.228277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-51843284-41a0-4a79-b76a-a872919db5e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed03e9a09f894d65b763130b332de271', 'request_id': 'req-51843284-41a0-4a79-b76a-a872919db5e8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:57.236933 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-51843284-41a0-4a79-b76a-a872919db5e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed03e9a09f894d65b763130b332de271', 'request_id': 'req-51843284-41a0-4a79-b76a-a872919db5e8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:57.249728 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-51843284-41a0-4a79-b76a-a872919db5e8 admin admin] The token cache is being invalidate because project ed03e9a09f894d65b763130b332de271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:00:57.250024 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-51843284-41a0-4a79-b76a-a872919db5e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:57.250675 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-51843284-41a0-4a79-b76a-a872919db5e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:00:57.252205 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1390/2801] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:00:57 2022] DELETE /identity/v3/projects/ed03e9a09f894d65b763130b332de271 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:57.257280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2f215-d93e-4176-afbf-c3ca815c884d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:57.257568 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2f215-d93e-4176-afbf-c3ca815c884d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:57.257842 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2f215-d93e-4176-afbf-c3ca815c884d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:57.288460 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e7c2f215-d93e-4176-afbf-c3ca815c884d None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:57.289773 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e7c2f215-d93e-4176-afbf-c3ca815c884d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:57.353635 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1412/2802] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:57.363654 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0d043-73af-4c52-a7bd-6b21594d7e58 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:57.363975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0d043-73af-4c52-a7bd-6b21594d7e58 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:57.364256 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0d043-73af-4c52-a7bd-6b21594d7e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:57.386068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-45c0d043-73af-4c52-a7bd-6b21594d7e58 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:57.387288 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-45c0d043-73af-4c52-a7bd-6b21594d7e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:57.395303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1391/2803] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:57.399753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2f215-d93e-4176-afbf-c3ca815c884d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:57.410804 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2f215-d93e-4176-afbf-c3ca815c884d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:57.415622 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b2c1d-4d62-4056-b8dc-47473cd6ce3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:57.416269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b2c1d-4d62-4056-b8dc-47473cd6ce3a admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:57.416548 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b2c1d-4d62-4056-b8dc-47473cd6ce3a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:57.416817 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b2c1d-4d62-4056-b8dc-47473cd6ce3a admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:57.417458 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b2c1d-4d62-4056-b8dc-47473cd6ce3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:00:57.417754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b2c1d-4d62-4056-b8dc-47473cd6ce3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:57.419433 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b2c1d-4d62-4056-b8dc-47473cd6ce3a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:57.431812 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1413/2804] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 15:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:00:57.437322 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0d043-73af-4c52-a7bd-6b21594d7e58 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:57.446659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0d043-73af-4c52-a7bd-6b21594d7e58 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:57.451851 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625bc6b-01f4-4a42-9327-93aa4d2f9130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:57.452659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625bc6b-01f4-4a42-9327-93aa4d2f9130 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:57.452940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625bc6b-01f4-4a42-9327-93aa4d2f9130 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:57.453400 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625bc6b-01f4-4a42-9327-93aa4d2f9130 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:57.453864 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625bc6b-01f4-4a42-9327-93aa4d2f9130 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:57.455763 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625bc6b-01f4-4a42-9327-93aa4d2f9130 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:57.485804 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1392/2805] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:00:57 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 15:00:57.490071 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b2c1d-4d62-4056-b8dc-47473cd6ce3a admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:57.501471 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b2c1d-4d62-4056-b8dc-47473cd6ce3a admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:57.561298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472ea5c-4946-4ffd-9844-aed901b5d83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:57.561873 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472ea5c-4946-4ffd-9844-aed901b5d83d admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:57.562156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472ea5c-4946-4ffd-9844-aed901b5d83d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:57.562431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b472ea5c-4946-4ffd-9844-aed901b5d83d admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:57.563067 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472ea5c-4946-4ffd-9844-aed901b5d83d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:57.565063 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b472ea5c-4946-4ffd-9844-aed901b5d83d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:57.586445 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1414/2806] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:00:57 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:57.591177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625bc6b-01f4-4a42-9327-93aa4d2f9130 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:57.611537 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625bc6b-01f4-4a42-9327-93aa4d2f9130 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:57.617981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea201a0d-6ff6-4efd-9705-ee324c3afdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:57.618695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea201a0d-6ff6-4efd-9705-ee324c3afdf7 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:57.619015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea201a0d-6ff6-4efd-9705-ee324c3afdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:57.619340 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea201a0d-6ff6-4efd-9705-ee324c3afdf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:57.620092 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea201a0d-6ff6-4efd-9705-ee324c3afdf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:57.621805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea201a0d-6ff6-4efd-9705-ee324c3afdf7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:57.632500 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1393/2807] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:57.637005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472ea5c-4946-4ffd-9844-aed901b5d83d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:57.649534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b472ea5c-4946-4ffd-9844-aed901b5d83d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:57.655486 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682b60f-a400-4502-a334-d3a2e0503da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:57.656588 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682b60f-a400-4502-a334-d3a2e0503da4 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:57.657110 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682b60f-a400-4502-a334-d3a2e0503da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:57.657611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682b60f-a400-4502-a334-d3a2e0503da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:57.658401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682b60f-a400-4502-a334-d3a2e0503da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:57.661577 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5682b60f-a400-4502-a334-d3a2e0503da4 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:57.676503 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1415/2808] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:57.680880 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea201a0d-6ff6-4efd-9705-ee324c3afdf7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:57.692031 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea201a0d-6ff6-4efd-9705-ee324c3afdf7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:57.696986 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96d649-08d0-4fa4-bfb4-9e22da7db50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:57.697692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96d649-08d0-4fa4-bfb4-9e22da7db50e admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:57.697969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96d649-08d0-4fa4-bfb4-9e22da7db50e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:57.698266 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f96d649-08d0-4fa4-bfb4-9e22da7db50e admin admin] PATH_INFO: `/v3/projects/56f8efa8a4b64b8ea073275d6b000fad/users/860502cfb4b4461db18f8ff440106ca5/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:57.706921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:57.717280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96d649-08d0-4fa4-bfb4-9e22da7db50e admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f8efa8a4b64b8ea073275d6b000fad, user_id=860502cfb4b4461db18f8ff440106ca5, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:57.720144 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f96d649-08d0-4fa4-bfb4-9e22da7db50e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:57.735313 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1394/2809] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:57 2022] PUT /identity/v3/projects/56f8efa8a4b64b8ea073275d6b000fad/users/860502cfb4b4461db18f8ff440106ca5/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:57.745855 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f277cc5-0020-4366-b231-ae5c89c3bc7d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:57.746143 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f277cc5-0020-4366-b231-ae5c89c3bc7d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:57.746442 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f277cc5-0020-4366-b231-ae5c89c3bc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:57.754662 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:57.777303 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-9f277cc5-0020-4366-b231-ae5c89c3bc7d None None] MFA Rules not processed for user `860502cfb4b4461db18f8ff440106ca5`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:57.778526 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9f277cc5-0020-4366-b231-ae5c89c3bc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:57.853176 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1416/2810] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:57.858527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96d649-08d0-4fa4-bfb4-9e22da7db50e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:57.927607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f96d649-08d0-4fa4-bfb4-9e22da7db50e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:57.932605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262ef62-d9e7-4c5f-a9dc-8b0714f118a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:57.933218 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262ef62-d9e7-4c5f-a9dc-8b0714f118a3 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:57.933497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262ef62-d9e7-4c5f-a9dc-8b0714f118a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:57.933829 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262ef62-d9e7-4c5f-a9dc-8b0714f118a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:57.934465 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262ef62-d9e7-4c5f-a9dc-8b0714f118a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:57.936686 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262ef62-d9e7-4c5f-a9dc-8b0714f118a3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:57.964334 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1395/2811] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:00:57 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:57.969166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f277cc5-0020-4366-b231-ae5c89c3bc7d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:57.980316 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f277cc5-0020-4366-b231-ae5c89c3bc7d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:58.007050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262ef62-d9e7-4c5f-a9dc-8b0714f118a3 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:58.009881 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d262ef62-d9e7-4c5f-a9dc-8b0714f118a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:58.056691 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3855f-1b06-410e-959f-7db80c050c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:58.057317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3855f-1b06-410e-959f-7db80c050c69 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:58.057919 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3855f-1b06-410e-959f-7db80c050c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:58.058195 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed3855f-1b06-410e-959f-7db80c050c69 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:58.058832 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3855f-1b06-410e-959f-7db80c050c69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:58.060824 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed3855f-1b06-410e-959f-7db80c050c69 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:58.082756 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262ef62-d9e7-4c5f-a9dc-8b0714f118a3 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:58.086723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98575b-0be6-4762-ba81-3523017bc801 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:58.087257 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1417/2812] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:00:57 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:58.088367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98575b-0be6-4762-ba81-3523017bc801 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:58.090839 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98575b-0be6-4762-ba81-3523017bc801 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:58.090944 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3855f-1b06-410e-959f-7db80c050c69 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:58.091078 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98575b-0be6-4762-ba81-3523017bc801 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:58.093576 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6d98575b-0be6-4762-ba81-3523017bc801 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:58.102287 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed3855f-1b06-410e-959f-7db80c050c69 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:58.107370 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51676fe-d4f0-4dec-8abe-8ffa10fb9e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:58.108191 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51676fe-d4f0-4dec-8abe-8ffa10fb9e3d admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:58.108521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51676fe-d4f0-4dec-8abe-8ffa10fb9e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:58.108822 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51676fe-d4f0-4dec-8abe-8ffa10fb9e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:58.109512 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51676fe-d4f0-4dec-8abe-8ffa10fb9e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:58.111108 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51676fe-d4f0-4dec-8abe-8ffa10fb9e3d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:58.125013 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1418/2813] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 15:00:58.129119 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51676fe-d4f0-4dec-8abe-8ffa10fb9e3d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:58.140436 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51676fe-d4f0-4dec-8abe-8ffa10fb9e3d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:58.146148 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd1564-9918-42da-9af2-b443368d5d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:58.146769 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd1564-9918-42da-9af2-b443368d5d47 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:58.147052 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd1564-9918-42da-9af2-b443368d5d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:58.147340 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd1564-9918-42da-9af2-b443368d5d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:58.148019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd1564-9918-42da-9af2-b443368d5d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:58.149621 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd1564-9918-42da-9af2-b443368d5d47 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:58.161850 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1419/2814] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 15:00:58.166305 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd1564-9918-42da-9af2-b443368d5d47 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:58.167524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d98575b-0be6-4762-ba81-3523017bc801 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:58.169017 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d98575b-0be6-4762-ba81-3523017bc801 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:58.176774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd1564-9918-42da-9af2-b443368d5d47 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:58.178531 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1396/2815] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 15:00:58.181676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef74919-12a5-45d1-b5fa-23ac97f0b61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:58.182565 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef74919-12a5-45d1-b5fa-23ac97f0b61f admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:58.182852 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef74919-12a5-45d1-b5fa-23ac97f0b61f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:58.183121 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef74919-12a5-45d1-b5fa-23ac97f0b61f admin admin] PATH_INFO: `/v3/projects/cbbe47d7f49149dc8a52587c68c3031e/users/65bf2b99899441669e1b12c6c4d5278f/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:58.201948 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef74919-12a5-45d1-b5fa-23ac97f0b61f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbbe47d7f49149dc8a52587c68c3031e, user_id=65bf2b99899441669e1b12c6c4d5278f, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:58.204650 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef74919-12a5-45d1-b5fa-23ac97f0b61f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:58.218058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1420/2816] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:58 2022] PUT /identity/v3/projects/cbbe47d7f49149dc8a52587c68c3031e/users/65bf2b99899441669e1b12c6c4d5278f/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 15:00:58.228484 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291d419-8fbb-4dae-95c3-ff04e441877b None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:58.229023 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291d419-8fbb-4dae-95c3-ff04e441877b None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:58.229023 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291d419-8fbb-4dae-95c3-ff04e441877b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:58.260325 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-8291d419-8fbb-4dae-95c3-ff04e441877b None None] MFA Rules not processed for user `65bf2b99899441669e1b12c6c4d5278f`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:58.261561 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-8291d419-8fbb-4dae-95c3-ff04e441877b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:58.278714 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:58.346909 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1397/2817] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:58.363602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef74919-12a5-45d1-b5fa-23ac97f0b61f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:58.434166 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:58.457454 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef74919-12a5-45d1-b5fa-23ac97f0b61f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:58.462188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ed20b-3804-456b-a718-62640bf6af10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:58.462790 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ed20b-3804-456b-a718-62640bf6af10 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:58.463075 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ed20b-3804-456b-a718-62640bf6af10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:58.463353 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ed20b-3804-456b-a718-62640bf6af10 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:58.464021 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6ed20b-3804-456b-a718-62640bf6af10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:58.465933 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6ed20b-3804-456b-a718-62640bf6af10 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:58.494019 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1421/2818] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:00:58 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:58.498562 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291d419-8fbb-4dae-95c3-ff04e441877b None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:58.509217 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291d419-8fbb-4dae-95c3-ff04e441877b None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:58.568527 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418e7f9-4d12-4940-b03a-3cd6126f11af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:58.568914 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418e7f9-4d12-4940-b03a-3cd6126f11af admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:58.569197 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418e7f9-4d12-4940-b03a-3cd6126f11af admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:58.569470 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418e7f9-4d12-4940-b03a-3cd6126f11af admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:58.570105 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418e7f9-4d12-4940-b03a-3cd6126f11af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:58.572058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418e7f9-4d12-4940-b03a-3cd6126f11af admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:58.594591 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1398/2819] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:00:58 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:00:58.598518 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ed20b-3804-456b-a718-62640bf6af10 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:58.609489 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ed20b-3804-456b-a718-62640bf6af10 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:58.614165 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1feb2aa-6b5b-4cbd-a6de-976261e53509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:58.614771 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1feb2aa-6b5b-4cbd-a6de-976261e53509 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:58.615055 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1feb2aa-6b5b-4cbd-a6de-976261e53509 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:58.615331 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1feb2aa-6b5b-4cbd-a6de-976261e53509 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:58.616003 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1feb2aa-6b5b-4cbd-a6de-976261e53509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:58.617596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1feb2aa-6b5b-4cbd-a6de-976261e53509 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:58.628693 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1422/2820] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:58.633108 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418e7f9-4d12-4940-b03a-3cd6126f11af admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:58.644703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418e7f9-4d12-4940-b03a-3cd6126f11af admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:58.649497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317abe9-a08d-4417-9c8d-a15dc64fec07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:58.650189 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317abe9-a08d-4417-9c8d-a15dc64fec07 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:58.650473 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317abe9-a08d-4417-9c8d-a15dc64fec07 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:58.650757 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317abe9-a08d-4417-9c8d-a15dc64fec07 admin admin] PATH_INFO: `/v3/projects/abe93afcddea4b0ea1d3409b968acf77/users/bbe8aa2aeb8b42c5bd1b209ddfb9e0c2/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:58.665576 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317abe9-a08d-4417-9c8d-a15dc64fec07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe93afcddea4b0ea1d3409b968acf77, user_id=bbe8aa2aeb8b42c5bd1b209ddfb9e0c2, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:58.668277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317abe9-a08d-4417-9c8d-a15dc64fec07 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:58.681743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1399/2821] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:58 2022] PUT /identity/v3/projects/abe93afcddea4b0ea1d3409b968acf77/users/bbe8aa2aeb8b42c5bd1b209ddfb9e0c2/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:58.687259 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1feb2aa-6b5b-4cbd-a6de-976261e53509 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:58.697558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1feb2aa-6b5b-4cbd-a6de-976261e53509 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:58.755061 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee93478-eeb9-4ff9-b84a-993978df615b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:58.755583 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee93478-eeb9-4ff9-b84a-993978df615b admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:58.755895 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee93478-eeb9-4ff9-b84a-993978df615b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:58.756171 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee93478-eeb9-4ff9-b84a-993978df615b admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:58.756890 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee93478-eeb9-4ff9-b84a-993978df615b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:58.758387 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee93478-eeb9-4ff9-b84a-993978df615b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:58.768046 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1423/2822] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:00:58.772146 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317abe9-a08d-4417-9c8d-a15dc64fec07 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:58.782345 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317abe9-a08d-4417-9c8d-a15dc64fec07 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:58.786925 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7e987-faf6-403d-9c10-d852ac3fdd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:58.787565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e987-faf6-403d-9c10-d852ac3fdd3d admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:58.787879 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e987-faf6-403d-9c10-d852ac3fdd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:58.788164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7e987-faf6-403d-9c10-d852ac3fdd3d admin admin] PATH_INFO: `/v3/projects/abe93afcddea4b0ea1d3409b968acf77/users/bbe8aa2aeb8b42c5bd1b209ddfb9e0c2/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:58.790056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7e987-faf6-403d-9c10-d852ac3fdd3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe93afcddea4b0ea1d3409b968acf77, user_id=bbe8aa2aeb8b42c5bd1b209ddfb9e0c2, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:58.792711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7e987-faf6-403d-9c10-d852ac3fdd3d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:58.803295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1400/2823] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:00:58 2022] PUT /identity/v3/projects/abe93afcddea4b0ea1d3409b968acf77/users/bbe8aa2aeb8b42c5bd1b209ddfb9e0c2/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:00:58.869847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed11a0-79e2-4dd7-ba5a-b2bdf723ec50 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:58.870141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed11a0-79e2-4dd7-ba5a-b2bdf723ec50 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:58.870421 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed11a0-79e2-4dd7-ba5a-b2bdf723ec50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:58.902931 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a1ed11a0-79e2-4dd7-ba5a-b2bdf723ec50 None None] MFA Rules not processed for user `bbe8aa2aeb8b42c5bd1b209ddfb9e0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:00:58.904362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a1ed11a0-79e2-4dd7-ba5a-b2bdf723ec50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:58.972690 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1424/2824] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:00:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:00:58.984380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7e987-faf6-403d-9c10-d852ac3fdd3d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:00:58.986087 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d5f7e987-faf6-403d-9c10-d852ac3fdd3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:00:59.043204 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7e987-faf6-403d-9c10-d852ac3fdd3d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:00:59.047495 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089fd39-476b-4549-ae62-67bcd68f1825 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:00:59.048135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089fd39-476b-4549-ae62-67bcd68f1825 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:00:59.048419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089fd39-476b-4549-ae62-67bcd68f1825 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:00:59.048692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089fd39-476b-4549-ae62-67bcd68f1825 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:00:59.069221 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:00:59.109156 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089fd39-476b-4549-ae62-67bcd68f1825 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:00:59.110595 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089fd39-476b-4549-ae62-67bcd68f1825 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:00:59.119526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1401/2825] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:00:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:01:02.754023 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed11a0-79e2-4dd7-ba5a-b2bdf723ec50 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:02.764966 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed11a0-79e2-4dd7-ba5a-b2bdf723ec50 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:02.768600 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c0520-1ba8-4a4f-96b9-be360a356663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:02.769199 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c0520-1ba8-4a4f-96b9-be360a356663 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:02.769474 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c0520-1ba8-4a4f-96b9-be360a356663 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:02.769740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c0520-1ba8-4a4f-96b9-be360a356663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:02.804563 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:02.842875 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c0520-1ba8-4a4f-96b9-be360a356663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:02.844372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c0520-1ba8-4a4f-96b9-be360a356663 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:02.854291 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1425/2826] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:01:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:01:03.429623 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089fd39-476b-4549-ae62-67bcd68f1825 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:03.438038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089fd39-476b-4549-ae62-67bcd68f1825 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:03.441637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54a409-7b09-41a8-a231-a185dd7afcef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:03.442225 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54a409-7b09-41a8-a231-a185dd7afcef service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:03.442502 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54a409-7b09-41a8-a231-a185dd7afcef service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:03.442771 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54a409-7b09-41a8-a231-a185dd7afcef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:03.452102 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54a409-7b09-41a8-a231-a185dd7afcef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:03.453667 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54a409-7b09-41a8-a231-a185dd7afcef service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:03.464651 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1402/2827] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:01:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:01:06.150822 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f637c-a1d2-4036-937a-b82f22abe7c8 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:06.151393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f637c-a1d2-4036-937a-b82f22abe7c8 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:06.151393 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f637c-a1d2-4036-937a-b82f22abe7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:06.172204 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5d0f637c-a1d2-4036-937a-b82f22abe7c8 None None] MFA Rules not processed for user `52173f24f42a481380b047abe76bca57`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:06.173432 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5d0f637c-a1d2-4036-937a-b82f22abe7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:06.233420 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1426/2828] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:08.505413 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54a409-7b09-41a8-a231-a185dd7afcef service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:08.598168 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54a409-7b09-41a8-a231-a185dd7afcef service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:08.604670 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfd620-072c-450f-8241-253528a84caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:08.605427 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfd620-072c-450f-8241-253528a84caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:08.605719 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfd620-072c-450f-8241-253528a84caf admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:08.606054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfd620-072c-450f-8241-253528a84caf admin admin] PATH_INFO: `/v3/users/860502cfb4b4461db18f8ff440106ca5` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:08.607507 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bfd620-072c-450f-8241-253528a84caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=860502cfb4b4461db18f8ff440106ca5)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:08.609976 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bfd620-072c-450f-8241-253528a84caf admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:08.686849 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-42bfd620-072c-450f-8241-253528a84caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '860502cfb4b4461db18f8ff440106ca5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:08.699503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-42bfd620-072c-450f-8241-253528a84caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '860502cfb4b4461db18f8ff440106ca5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:08.722895 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-42bfd620-072c-450f-8241-253528a84caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '860502cfb4b4461db18f8ff440106ca5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:08.735942 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-42bfd620-072c-450f-8241-253528a84caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '860502cfb4b4461db18f8ff440106ca5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:08.736697 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-42bfd620-072c-450f-8241-253528a84caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '860502cfb4b4461db18f8ff440106ca5'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:08.760312 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1403/2829] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:01:08 2022] DELETE /identity/v3/users/860502cfb4b4461db18f8ff440106ca5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:08.760444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991d6b5-3d5b-4693-b0ad-4f0a32239037 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:08.760444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991d6b5-3d5b-4693-b0ad-4f0a32239037 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:08.760444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991d6b5-3d5b-4693-b0ad-4f0a32239037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:08.784624 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b991d6b5-3d5b-4693-b0ad-4f0a32239037 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:08.785998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b991d6b5-3d5b-4693-b0ad-4f0a32239037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:08.867874 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1427/2830] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:08.875981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfd620-072c-450f-8241-253528a84caf admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:08.877526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-42bfd620-072c-450f-8241-253528a84caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:08.935848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:08.978957 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfd620-072c-450f-8241-253528a84caf admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:08.984525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086bc59-a317-46fb-b8b4-925fa8f5f056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:08.985163 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086bc59-a317-46fb-b8b4-925fa8f5f056 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:08.985445 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086bc59-a317-46fb-b8b4-925fa8f5f056 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:08.985723 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086bc59-a317-46fb-b8b4-925fa8f5f056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:08.996498 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086bc59-a317-46fb-b8b4-925fa8f5f056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:08.998347 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086bc59-a317-46fb-b8b4-925fa8f5f056 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:09.009582 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1404/2831] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:01:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:01:09.413992 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991d6b5-3d5b-4693-b0ad-4f0a32239037 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:09.415480 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b991d6b5-3d5b-4693-b0ad-4f0a32239037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:09.430364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991d6b5-3d5b-4693-b0ad-4f0a32239037 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:09.436853 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c1043-f714-4d60-94ea-4852ddcd866e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:09.440534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c1043-f714-4d60-94ea-4852ddcd866e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:09.440871 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c1043-f714-4d60-94ea-4852ddcd866e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:09.443997 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c1043-f714-4d60-94ea-4852ddcd866e admin admin] PATH_INFO: `/v3/projects/56f8efa8a4b64b8ea073275d6b000fad` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:09.446671 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c1043-f714-4d60-94ea-4852ddcd866e admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f8efa8a4b64b8ea073275d6b000fad)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:09.447552 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c1043-f714-4d60-94ea-4852ddcd866e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:09.506762 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7e5c1043-f714-4d60-94ea-4852ddcd866e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f8efa8a4b64b8ea073275d6b000fad', 'request_id': 'req-7e5c1043-f714-4d60-94ea-4852ddcd866e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:09.513501 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7e5c1043-f714-4d60-94ea-4852ddcd866e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f8efa8a4b64b8ea073275d6b000fad', 'request_id': 'req-7e5c1043-f714-4d60-94ea-4852ddcd866e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:09.522665 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7e5c1043-f714-4d60-94ea-4852ddcd866e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f8efa8a4b64b8ea073275d6b000fad', 'request_id': 'req-7e5c1043-f714-4d60-94ea-4852ddcd866e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:09.534311 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:09.539080 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7e5c1043-f714-4d60-94ea-4852ddcd866e admin admin] The token cache is being invalidate because project 56f8efa8a4b64b8ea073275d6b000fad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:01:09.539620 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7e5c1043-f714-4d60-94ea-4852ddcd866e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:09.540960 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-7e5c1043-f714-4d60-94ea-4852ddcd866e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:09.542844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1428/2832] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:01:09 2022] DELETE /identity/v3/projects/56f8efa8a4b64b8ea073275d6b000fad => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:09.547773 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086bc59-a317-46fb-b8b4-925fa8f5f056 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:09.560738 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086bc59-a317-46fb-b8b4-925fa8f5f056 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:09.635660 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348abf9-f83d-48b7-8410-b9328c5179be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:09.636380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348abf9-f83d-48b7-8410-b9328c5179be admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:09.636704 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348abf9-f83d-48b7-8410-b9328c5179be admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:09.636984 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348abf9-f83d-48b7-8410-b9328c5179be admin admin] PATH_INFO: `/v3/users/65bf2b99899441669e1b12c6c4d5278f` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:09.638802 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348abf9-f83d-48b7-8410-b9328c5179be admin admin] RBAC: Authorizing `identity:delete_user(user_id=65bf2b99899441669e1b12c6c4d5278f)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:09.640927 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c348abf9-f83d-48b7-8410-b9328c5179be admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:09.706706 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c348abf9-f83d-48b7-8410-b9328c5179be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65bf2b99899441669e1b12c6c4d5278f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:09.720891 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c348abf9-f83d-48b7-8410-b9328c5179be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65bf2b99899441669e1b12c6c4d5278f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:09.736333 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c348abf9-f83d-48b7-8410-b9328c5179be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65bf2b99899441669e1b12c6c4d5278f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:09.749906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c348abf9-f83d-48b7-8410-b9328c5179be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65bf2b99899441669e1b12c6c4d5278f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:09.750540 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-c348abf9-f83d-48b7-8410-b9328c5179be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65bf2b99899441669e1b12c6c4d5278f'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:09.757943 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1405/2833] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:01:09 2022] DELETE /identity/v3/users/65bf2b99899441669e1b12c6c4d5278f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:10.068210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c1043-f714-4d60-94ea-4852ddcd866e admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:10.069740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-7e5c1043-f714-4d60-94ea-4852ddcd866e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:10.086444 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c1043-f714-4d60-94ea-4852ddcd866e admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:10.169032 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d39ea7-966b-49fe-9641-b15327dbe041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:10.170016 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d39ea7-966b-49fe-9641-b15327dbe041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:10.170535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d39ea7-966b-49fe-9641-b15327dbe041 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:10.173210 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d39ea7-966b-49fe-9641-b15327dbe041 admin admin] PATH_INFO: `/v3/projects/cbbe47d7f49149dc8a52587c68c3031e` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:10.174517 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d39ea7-966b-49fe-9641-b15327dbe041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbbe47d7f49149dc8a52587c68c3031e)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:10.176620 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d39ea7-966b-49fe-9641-b15327dbe041 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:10.230051 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a8d39ea7-966b-49fe-9641-b15327dbe041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbbe47d7f49149dc8a52587c68c3031e', 'request_id': 'req-a8d39ea7-966b-49fe-9641-b15327dbe041'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:10.236598 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a8d39ea7-966b-49fe-9641-b15327dbe041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbbe47d7f49149dc8a52587c68c3031e', 'request_id': 'req-a8d39ea7-966b-49fe-9641-b15327dbe041'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:10.244306 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a8d39ea7-966b-49fe-9641-b15327dbe041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbbe47d7f49149dc8a52587c68c3031e', 'request_id': 'req-a8d39ea7-966b-49fe-9641-b15327dbe041'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:10.279585 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a8d39ea7-966b-49fe-9641-b15327dbe041 admin admin] The token cache is being invalidate because project cbbe47d7f49149dc8a52587c68c3031e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:01:10.280104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a8d39ea7-966b-49fe-9641-b15327dbe041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:10.281113 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a8d39ea7-966b-49fe-9641-b15327dbe041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:10.285834 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1429/2834] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:01:10 2022] DELETE /identity/v3/projects/cbbe47d7f49149dc8a52587c68c3031e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:10.290452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348abf9-f83d-48b7-8410-b9328c5179be admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:10.292719 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c348abf9-f83d-48b7-8410-b9328c5179be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:10.312018 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348abf9-f83d-48b7-8410-b9328c5179be admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:10.392908 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:10.413278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582eda41-7c03-44bb-8dd9-73f04e44edd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:10.413936 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582eda41-7c03-44bb-8dd9-73f04e44edd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:10.414225 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582eda41-7c03-44bb-8dd9-73f04e44edd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:10.414505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582eda41-7c03-44bb-8dd9-73f04e44edd2 admin admin] PATH_INFO: `/v3/users/bbe8aa2aeb8b42c5bd1b209ddfb9e0c2` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:10.416027 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582eda41-7c03-44bb-8dd9-73f04e44edd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbe8aa2aeb8b42c5bd1b209ddfb9e0c2)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:10.418193 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582eda41-7c03-44bb-8dd9-73f04e44edd2 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:10.472736 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-582eda41-7c03-44bb-8dd9-73f04e44edd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbe8aa2aeb8b42c5bd1b209ddfb9e0c2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:10.480452 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-582eda41-7c03-44bb-8dd9-73f04e44edd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbe8aa2aeb8b42c5bd1b209ddfb9e0c2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:10.494975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-582eda41-7c03-44bb-8dd9-73f04e44edd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbe8aa2aeb8b42c5bd1b209ddfb9e0c2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:10.504008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-582eda41-7c03-44bb-8dd9-73f04e44edd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbe8aa2aeb8b42c5bd1b209ddfb9e0c2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:10.504633 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-582eda41-7c03-44bb-8dd9-73f04e44edd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbe8aa2aeb8b42c5bd1b209ddfb9e0c2'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:10.511095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1406/2835] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:01:10 2022] DELETE /identity/v3/users/bbe8aa2aeb8b42c5bd1b209ddfb9e0c2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 15:01:10.948062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d39ea7-966b-49fe-9641-b15327dbe041 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:10.950072 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a8d39ea7-966b-49fe-9641-b15327dbe041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:10.978990 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d39ea7-966b-49fe-9641-b15327dbe041 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:11.042880 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0f0f7-aed2-4dfd-aebd-25e83ff5a65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:11.043505 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0f0f7-aed2-4dfd-aebd-25e83ff5a65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:11.043785 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0f0f7-aed2-4dfd-aebd-25e83ff5a65c admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:11.044090 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0f0f7-aed2-4dfd-aebd-25e83ff5a65c admin admin] PATH_INFO: `/v3/projects/abe93afcddea4b0ea1d3409b968acf77` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:11.045786 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0f0f7-aed2-4dfd-aebd-25e83ff5a65c admin admin] RBAC: Authorizing `identity:delete_project(project_id=abe93afcddea4b0ea1d3409b968acf77)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:11.047797 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0f0f7-aed2-4dfd-aebd-25e83ff5a65c admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:11.093254 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:11.108624 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bdb0f0f7-aed2-4dfd-aebd-25e83ff5a65c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abe93afcddea4b0ea1d3409b968acf77', 'request_id': 'req-bdb0f0f7-aed2-4dfd-aebd-25e83ff5a65c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:11.114892 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bdb0f0f7-aed2-4dfd-aebd-25e83ff5a65c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abe93afcddea4b0ea1d3409b968acf77', 'request_id': 'req-bdb0f0f7-aed2-4dfd-aebd-25e83ff5a65c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:11.121689 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bdb0f0f7-aed2-4dfd-aebd-25e83ff5a65c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abe93afcddea4b0ea1d3409b968acf77', 'request_id': 'req-bdb0f0f7-aed2-4dfd-aebd-25e83ff5a65c'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:11.136646 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bdb0f0f7-aed2-4dfd-aebd-25e83ff5a65c admin admin] The token cache is being invalidate because project abe93afcddea4b0ea1d3409b968acf77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:01:11.136935 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bdb0f0f7-aed2-4dfd-aebd-25e83ff5a65c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:11.139893 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-bdb0f0f7-aed2-4dfd-aebd-25e83ff5a65c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:11.141679 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1430/2836] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:01:10 2022] DELETE /identity/v3/projects/abe93afcddea4b0ea1d3409b968acf77 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:11.147380 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5fbc4-228f-48ad-89b5-cebec5122aa7 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:11.147860 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5fbc4-228f-48ad-89b5-cebec5122aa7 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:11.147979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5fbc4-228f-48ad-89b5-cebec5122aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:11.172308 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-a2e5fbc4-228f-48ad-89b5-cebec5122aa7 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:11.173549 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a2e5fbc4-228f-48ad-89b5-cebec5122aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:11.241314 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1407/2837] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:11.249401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dae194-e329-424b-83ff-957ad1b6e361 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:11.249969 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dae194-e329-424b-83ff-957ad1b6e361 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:11.250340 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dae194-e329-424b-83ff-957ad1b6e361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:11.271776 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b4dae194-e329-424b-83ff-957ad1b6e361 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:11.274156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b4dae194-e329-424b-83ff-957ad1b6e361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:11.283233 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1431/2838] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:11.287590 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5fbc4-228f-48ad-89b5-cebec5122aa7 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:11.298016 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5fbc4-228f-48ad-89b5-cebec5122aa7 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:11.303057 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80bb93-faa4-44d9-af2a-59a3b796276d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:11.303710 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80bb93-faa4-44d9-af2a-59a3b796276d admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:11.304281 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80bb93-faa4-44d9-af2a-59a3b796276d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:11.304545 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80bb93-faa4-44d9-af2a-59a3b796276d admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:11.305050 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80bb93-faa4-44d9-af2a-59a3b796276d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:01:11.305367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80bb93-faa4-44d9-af2a-59a3b796276d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:11.306984 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80bb93-faa4-44d9-af2a-59a3b796276d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:11.319791 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1408/2839] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 15:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:01:11.324153 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dae194-e329-424b-83ff-957ad1b6e361 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:11.334343 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dae194-e329-424b-83ff-957ad1b6e361 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:11.339246 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda0561-62ca-41b2-99da-204d7b64b0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:11.339886 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda0561-62ca-41b2-99da-204d7b64b0a7 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:11.340167 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda0561-62ca-41b2-99da-204d7b64b0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:11.340439 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda0561-62ca-41b2-99da-204d7b64b0a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:11.341132 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda0561-62ca-41b2-99da-204d7b64b0a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:11.343608 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda0561-62ca-41b2-99da-204d7b64b0a7 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:11.371610 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1432/2840] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:01:11 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:01:11.375913 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80bb93-faa4-44d9-af2a-59a3b796276d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:11.385940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80bb93-faa4-44d9-af2a-59a3b796276d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:11.440855 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:11.449780 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753b8ea-fdbf-43c9-bddb-6848bf4debfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:11.451145 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753b8ea-fdbf-43c9-bddb-6848bf4debfa admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:11.451457 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753b8ea-fdbf-43c9-bddb-6848bf4debfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:11.452008 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753b8ea-fdbf-43c9-bddb-6848bf4debfa admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:11.452911 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753b8ea-fdbf-43c9-bddb-6848bf4debfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:11.455577 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753b8ea-fdbf-43c9-bddb-6848bf4debfa admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:11.482593 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1409/2841] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:01:11 2022] POST /identity/v3/users => generated 284 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:01:11.483915 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda0561-62ca-41b2-99da-204d7b64b0a7 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:11.494395 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda0561-62ca-41b2-99da-204d7b64b0a7 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:11.499708 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f463f-a760-40ee-a1b2-c54a426d0f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:11.500401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f463f-a760-40ee-a1b2-c54a426d0f9b admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:11.500688 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f463f-a760-40ee-a1b2-c54a426d0f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:11.500964 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f463f-a760-40ee-a1b2-c54a426d0f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:11.501592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f463f-a760-40ee-a1b2-c54a426d0f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:11.503216 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f463f-a760-40ee-a1b2-c54a426d0f9b admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:11.513998 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1433/2842] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 15:01:11.518137 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753b8ea-fdbf-43c9-bddb-6848bf4debfa admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:11.529692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753b8ea-fdbf-43c9-bddb-6848bf4debfa admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:11.534486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4068240-80c2-48e2-9119-b3db4ae24420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:11.535038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4068240-80c2-48e2-9119-b3db4ae24420 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:11.535321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4068240-80c2-48e2-9119-b3db4ae24420 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:11.535696 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4068240-80c2-48e2-9119-b3db4ae24420 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:11.536252 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4068240-80c2-48e2-9119-b3db4ae24420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:11.541037 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4068240-80c2-48e2-9119-b3db4ae24420 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:11.553944 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1410/2843] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:01:11.558570 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f463f-a760-40ee-a1b2-c54a426d0f9b admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:11.569357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f463f-a760-40ee-a1b2-c54a426d0f9b admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:11.574031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc73bd-7d13-4879-a650-5a374e3d7b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:11.574698 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc73bd-7d13-4879-a650-5a374e3d7b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:11.574986 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc73bd-7d13-4879-a650-5a374e3d7b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:11.575265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc73bd-7d13-4879-a650-5a374e3d7b73 admin admin] PATH_INFO: `/v3/projects/3989c7cfaaaf46be8ccb361ba23a17e2/users/420f20f1b74742dfbe87cee54406600e/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:11.593626 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc73bd-7d13-4879-a650-5a374e3d7b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3989c7cfaaaf46be8ccb361ba23a17e2, user_id=420f20f1b74742dfbe87cee54406600e, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:11.596371 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc73bd-7d13-4879-a650-5a374e3d7b73 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:11.617754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1434/2844] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:01:11 2022] PUT /identity/v3/projects/3989c7cfaaaf46be8ccb361ba23a17e2/users/420f20f1b74742dfbe87cee54406600e/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:11.629239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a99eb-61b0-4d5c-96f5-08507828f893 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:11.629239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a99eb-61b0-4d5c-96f5-08507828f893 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:11.629484 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a99eb-61b0-4d5c-96f5-08507828f893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:11.664536 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-107a99eb-61b0-4d5c-96f5-08507828f893 None None] MFA Rules not processed for user `420f20f1b74742dfbe87cee54406600e`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:11.665754 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-107a99eb-61b0-4d5c-96f5-08507828f893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:11.729844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1411/2845] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 19 15:01:11.741669 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc73bd-7d13-4879-a650-5a374e3d7b73 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:11.743546 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-c8bc73bd-7d13-4879-a650-5a374e3d7b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:11.804534 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc73bd-7d13-4879-a650-5a374e3d7b73 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:11.811794 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6dbf8-7f48-4f06-80ac-0d00ce332d5d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caab7aaa72634b358d717768fbcb48d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:11.812363 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6dbf8-7f48-4f06-80ac-0d00ce332d5d service glance] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:11.812696 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6dbf8-7f48-4f06-80ac-0d00ce332d5d service glance] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:11.813009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6dbf8-7f48-4f06-80ac-0d00ce332d5d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:11.822256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6dbf8-7f48-4f06-80ac-0d00ce332d5d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:11.823996 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6dbf8-7f48-4f06-80ac-0d00ce332d5d service glance] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:11.834097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1435/2846] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:01:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 19 15:01:13.840693 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a99eb-61b0-4d5c-96f5-08507828f893 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:13.842244 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-107a99eb-61b0-4d5c-96f5-08507828f893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:13.912210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a99eb-61b0-4d5c-96f5-08507828f893 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:13.917219 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902c053-bd5d-4d92-808f-ed08f6c729cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:13.917566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902c053-bd5d-4d92-808f-ed08f6c729cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:13.917856 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902c053-bd5d-4d92-808f-ed08f6c729cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:13.918184 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902c053-bd5d-4d92-808f-ed08f6c729cc admin admin] PATH_INFO: `/v3/users/4c32ce76e5c145048983199431c13f23` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:13.919369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902c053-bd5d-4d92-808f-ed08f6c729cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c32ce76e5c145048983199431c13f23)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:13.921336 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2902c053-bd5d-4d92-808f-ed08f6c729cc admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:13.984855 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2902c053-bd5d-4d92-808f-ed08f6c729cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c32ce76e5c145048983199431c13f23'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:13.991807 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2902c053-bd5d-4d92-808f-ed08f6c729cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c32ce76e5c145048983199431c13f23'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:14.003748 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:14.009403 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2902c053-bd5d-4d92-808f-ed08f6c729cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c32ce76e5c145048983199431c13f23'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:14.018239 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2902c053-bd5d-4d92-808f-ed08f6c729cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c32ce76e5c145048983199431c13f23'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:14.018863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-2902c053-bd5d-4d92-808f-ed08f6c729cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c32ce76e5c145048983199431c13f23'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:14.026928 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1412/2847] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:01:13 2022] DELETE /identity/v3/users/4c32ce76e5c145048983199431c13f23 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:14.031779 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea747cd-2200-4288-9714-56232af9b0e0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:14.032327 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea747cd-2200-4288-9714-56232af9b0e0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:14.032466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea747cd-2200-4288-9714-56232af9b0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:14.053006 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-dea747cd-2200-4288-9714-56232af9b0e0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:14.054628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-dea747cd-2200-4288-9714-56232af9b0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:14.111767 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:14.125472 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1436/2848] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:14.132792 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902c053-bd5d-4d92-808f-ed08f6c729cc admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:14.134270 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-2902c053-bd5d-4d92-808f-ed08f6c729cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:14.199070 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902c053-bd5d-4d92-808f-ed08f6c729cc admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:14.203230 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15128dd3-138d-43ac-a770-b7725589798c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:14.203844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15128dd3-138d-43ac-a770-b7725589798c service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:14.204144 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15128dd3-138d-43ac-a770-b7725589798c service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:14.204414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15128dd3-138d-43ac-a770-b7725589798c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:14.215419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15128dd3-138d-43ac-a770-b7725589798c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:14.217433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15128dd3-138d-43ac-a770-b7725589798c service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:14.228031 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1413/2849] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:01:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:01:14.530477 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea747cd-2200-4288-9714-56232af9b0e0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:14.533881 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-dea747cd-2200-4288-9714-56232af9b0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:14.548680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea747cd-2200-4288-9714-56232af9b0e0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:14.555221 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344fed62-125c-4a79-bd37-5813d58c3b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:14.556271 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fed62-125c-4a79-bd37-5813d58c3b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:14.556719 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fed62-125c-4a79-bd37-5813d58c3b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:14.557224 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344fed62-125c-4a79-bd37-5813d58c3b1e admin admin] PATH_INFO: `/v3/projects/b62ee0da1ed84a1288bc9e8ccf377cbe` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:14.558870 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344fed62-125c-4a79-bd37-5813d58c3b1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b62ee0da1ed84a1288bc9e8ccf377cbe)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:14.562460 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344fed62-125c-4a79-bd37-5813d58c3b1e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:14.612801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-344fed62-125c-4a79-bd37-5813d58c3b1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b62ee0da1ed84a1288bc9e8ccf377cbe', 'request_id': 'req-344fed62-125c-4a79-bd37-5813d58c3b1e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:14.618604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-344fed62-125c-4a79-bd37-5813d58c3b1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b62ee0da1ed84a1288bc9e8ccf377cbe', 'request_id': 'req-344fed62-125c-4a79-bd37-5813d58c3b1e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:14.625280 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-344fed62-125c-4a79-bd37-5813d58c3b1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b62ee0da1ed84a1288bc9e8ccf377cbe', 'request_id': 'req-344fed62-125c-4a79-bd37-5813d58c3b1e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:14.639179 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-344fed62-125c-4a79-bd37-5813d58c3b1e admin admin] The token cache is being invalidate because project b62ee0da1ed84a1288bc9e8ccf377cbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:01:14.639450 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-344fed62-125c-4a79-bd37-5813d58c3b1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:14.640140 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-344fed62-125c-4a79-bd37-5813d58c3b1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:14.641612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1437/2850] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:01:14 2022] DELETE /identity/v3/projects/b62ee0da1ed84a1288bc9e8ccf377cbe => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:14.646979 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa85f7-e7c3-49dc-9a60-f3db7a9ce631 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:14.647266 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa85f7-e7c3-49dc-9a60-f3db7a9ce631 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:14.647544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfa85f7-e7c3-49dc-9a60-f3db7a9ce631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:14.672068 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ddfa85f7-e7c3-49dc-9a60-f3db7a9ce631 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:14.673923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ddfa85f7-e7c3-49dc-9a60-f3db7a9ce631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:14.740622 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1414/2851] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:14.750066 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e53f5-c142-441d-b691-752fa1ffc586 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:14.750438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e53f5-c142-441d-b691-752fa1ffc586 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:14.750729 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e53f5-c142-441d-b691-752fa1ffc586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:14.773725 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-ce5e53f5-c142-441d-b691-752fa1ffc586 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:14.775104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ce5e53f5-c142-441d-b691-752fa1ffc586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:14.784485 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1438/2852] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:14.790278 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa85f7-e7c3-49dc-9a60-f3db7a9ce631 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:14.801022 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfa85f7-e7c3-49dc-9a60-f3db7a9ce631 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:14.805597 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ccd30-f47e-4583-8562-a5ab9fef64b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:14.806191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ccd30-f47e-4583-8562-a5ab9fef64b7 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:14.806467 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ccd30-f47e-4583-8562-a5ab9fef64b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:14.806733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ccd30-f47e-4583-8562-a5ab9fef64b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:14.807610 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ccd30-f47e-4583-8562-a5ab9fef64b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:01:14.807953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ccd30-f47e-4583-8562-a5ab9fef64b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:14.809543 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ccd30-f47e-4583-8562-a5ab9fef64b7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:14.821658 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1415/2853] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 15:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:01:14.826236 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e53f5-c142-441d-b691-752fa1ffc586 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:14.838101 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e53f5-c142-441d-b691-752fa1ffc586 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:14.843062 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781c532-0b5f-4429-babd-b20594ad00fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:14.843663 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781c532-0b5f-4429-babd-b20594ad00fd admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:14.843993 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781c532-0b5f-4429-babd-b20594ad00fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:14.844265 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c781c532-0b5f-4429-babd-b20594ad00fd admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:14.844896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c781c532-0b5f-4429-babd-b20594ad00fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:14.846782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c781c532-0b5f-4429-babd-b20594ad00fd admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:14.876080 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1439/2854] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:01:14 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:01:14.880662 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ccd30-f47e-4583-8562-a5ab9fef64b7 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:14.892439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ccd30-f47e-4583-8562-a5ab9fef64b7 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:14.951480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822883c-06a3-4e35-b99b-2a18a461add6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:14.952199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822883c-06a3-4e35-b99b-2a18a461add6 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:14.952548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822883c-06a3-4e35-b99b-2a18a461add6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:14.952822 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822883c-06a3-4e35-b99b-2a18a461add6 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:14.953502 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822883c-06a3-4e35-b99b-2a18a461add6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:14.955518 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822883c-06a3-4e35-b99b-2a18a461add6 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:14.978354 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1416/2855] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:01:14 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:01:14.982980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781c532-0b5f-4429-babd-b20594ad00fd admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:14.994686 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c781c532-0b5f-4429-babd-b20594ad00fd admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:15.000364 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b99b7-ba07-4050-864b-0dff876c597f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:15.001054 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b99b7-ba07-4050-864b-0dff876c597f admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:15.001407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b99b7-ba07-4050-864b-0dff876c597f admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:15.001753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b99b7-ba07-4050-864b-0dff876c597f admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:15.002466 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b99b7-ba07-4050-864b-0dff876c597f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:15.004141 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b99b7-ba07-4050-864b-0dff876c597f admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:15.015076 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1440/2856] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:01:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:01:15.020226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822883c-06a3-4e35-b99b-2a18a461add6 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:15.034210 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:15.040822 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822883c-06a3-4e35-b99b-2a18a461add6 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:15.045118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f65212-cee6-446b-b74e-e89a3fdecfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:15.046060 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f65212-cee6-446b-b74e-e89a3fdecfd9 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:15.046533 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f65212-cee6-446b-b74e-e89a3fdecfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:15.047567 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f65212-cee6-446b-b74e-e89a3fdecfd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:15.048859 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f65212-cee6-446b-b74e-e89a3fdecfd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:15.051014 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f65212-cee6-446b-b74e-e89a3fdecfd9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:15.063443 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1417/2857] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:01:15.068161 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b99b7-ba07-4050-864b-0dff876c597f admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:15.080189 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b99b7-ba07-4050-864b-0dff876c597f admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:15.085567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cab452-0b8a-4728-805b-632609f74545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:15.086244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cab452-0b8a-4728-805b-632609f74545 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:15.086524 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cab452-0b8a-4728-805b-632609f74545 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:15.086798 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cab452-0b8a-4728-805b-632609f74545 admin admin] PATH_INFO: `/v3/projects/747b4e137fa1402eb44c1f50313ff530/users/ffa4461472f6424d8a1563ab2e059c4d/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:15.101889 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cab452-0b8a-4728-805b-632609f74545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=747b4e137fa1402eb44c1f50313ff530, user_id=ffa4461472f6424d8a1563ab2e059c4d, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:15.104549 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cab452-0b8a-4728-805b-632609f74545 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:15.120851 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1441/2858] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:01:15 2022] PUT /identity/v3/projects/747b4e137fa1402eb44c1f50313ff530/users/ffa4461472f6424d8a1563ab2e059c4d/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:15.133165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9117e9-1494-47ca-b4a0-bfbe1522c122 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:15.133457 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9117e9-1494-47ca-b4a0-bfbe1522c122 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:15.133741 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9117e9-1494-47ca-b4a0-bfbe1522c122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:15.163372 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-9e9117e9-1494-47ca-b4a0-bfbe1522c122 None None] MFA Rules not processed for user `ffa4461472f6424d8a1563ab2e059c4d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:15.164637 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9e9117e9-1494-47ca-b4a0-bfbe1522c122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:15.230054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1418/2859] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:15.241416 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cab452-0b8a-4728-805b-632609f74545 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:15.243238 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-87cab452-0b8a-4728-805b-632609f74545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:15.301206 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cab452-0b8a-4728-805b-632609f74545 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:15.304842 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b005c-4682-4617-a79a-4afacc2355e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:15.305455 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b005c-4682-4617-a79a-4afacc2355e0 service cinder] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:15.305737 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b005c-4682-4617-a79a-4afacc2355e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:15.306010 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b005c-4682-4617-a79a-4afacc2355e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:15.315104 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b005c-4682-4617-a79a-4afacc2355e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:15.316586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b005c-4682-4617-a79a-4afacc2355e0 service cinder] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:15.325896 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1442/2860] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:01:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:01:25.227312 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=35 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:25.286366 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9117e9-1494-47ca-b4a0-bfbe1522c122 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:25.287863 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9e9117e9-1494-47ca-b4a0-bfbe1522c122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:25.321586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:25.346650 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9117e9-1494-47ca-b4a0-bfbe1522c122 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:25.350275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5d50b-a693-488c-8a2b-fd711ba73728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:25.350886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5d50b-a693-488c-8a2b-fd711ba73728 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:25.351166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5d50b-a693-488c-8a2b-fd711ba73728 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:25.351436 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5d50b-a693-488c-8a2b-fd711ba73728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:25.353165 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6fb5d50b-a693-488c-8a2b-fd711ba73728 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:25.416608 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5d50b-a693-488c-8a2b-fd711ba73728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:25.418056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5d50b-a693-488c-8a2b-fd711ba73728 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:25.431045 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1419/2861] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:01:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:01:25.552820 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b005c-4682-4617-a79a-4afacc2355e0 service cinder] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:25.618947 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b005c-4682-4617-a79a-4afacc2355e0 service cinder] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:25.623943 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258a0e8-e597-4b15-ba03-f8e772eadfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:25.624604 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258a0e8-e597-4b15-ba03-f8e772eadfd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:25.624934 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258a0e8-e597-4b15-ba03-f8e772eadfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:25.625169 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258a0e8-e597-4b15-ba03-f8e772eadfd5 admin admin] PATH_INFO: `/v3/users/ffa4461472f6424d8a1563ab2e059c4d` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:25.626292 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258a0e8-e597-4b15-ba03-f8e772eadfd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffa4461472f6424d8a1563ab2e059c4d)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:25.628310 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258a0e8-e597-4b15-ba03-f8e772eadfd5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:25.680254 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1258a0e8-e597-4b15-ba03-f8e772eadfd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffa4461472f6424d8a1563ab2e059c4d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:25.688127 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1258a0e8-e597-4b15-ba03-f8e772eadfd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffa4461472f6424d8a1563ab2e059c4d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:25.702294 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1258a0e8-e597-4b15-ba03-f8e772eadfd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffa4461472f6424d8a1563ab2e059c4d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:25.709742 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1258a0e8-e597-4b15-ba03-f8e772eadfd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffa4461472f6424d8a1563ab2e059c4d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:25.710418 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1258a0e8-e597-4b15-ba03-f8e772eadfd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffa4461472f6424d8a1563ab2e059c4d'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:25.718401 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1443/2862] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:01:25 2022] DELETE /identity/v3/users/ffa4461472f6424d8a1563ab2e059c4d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:25.723148 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03539271-775d-4eae-bb01-b03b3da28dd4 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:25.723450 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03539271-775d-4eae-bb01-b03b3da28dd4 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:25.723746 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03539271-775d-4eae-bb01-b03b3da28dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:25.744149 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-03539271-775d-4eae-bb01-b03b3da28dd4 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:25.745526 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-03539271-775d-4eae-bb01-b03b3da28dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:25.810192 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1420/2863] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:25.819956 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258a0e8-e597-4b15-ba03-f8e772eadfd5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:25.821407 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1258a0e8-e597-4b15-ba03-f8e772eadfd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:25.882227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258a0e8-e597-4b15-ba03-f8e772eadfd5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:25.887759 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca66f70-849e-4836-9ad4-ba6ce452e53c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:25.888482 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca66f70-849e-4836-9ad4-ba6ce452e53c service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:25.888754 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca66f70-849e-4836-9ad4-ba6ce452e53c service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:25.889096 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca66f70-849e-4836-9ad4-ba6ce452e53c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:25.901252 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca66f70-849e-4836-9ad4-ba6ce452e53c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:25.902703 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca66f70-849e-4836-9ad4-ba6ce452e53c service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:25.913988 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1444/2864] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:01:26.214433 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03539271-775d-4eae-bb01-b03b3da28dd4 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:26.216968 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-03539271-775d-4eae-bb01-b03b3da28dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:26.230213 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03539271-775d-4eae-bb01-b03b3da28dd4 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:26.237225 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de7074-e5d4-4d10-bfbc-dc1992ae6399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:26.238056 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de7074-e5d4-4d10-bfbc-dc1992ae6399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:26.238328 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de7074-e5d4-4d10-bfbc-dc1992ae6399 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:26.238607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de7074-e5d4-4d10-bfbc-dc1992ae6399 admin admin] PATH_INFO: `/v3/projects/747b4e137fa1402eb44c1f50313ff530` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:26.239813 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de7074-e5d4-4d10-bfbc-dc1992ae6399 admin admin] RBAC: Authorizing `identity:delete_project(project_id=747b4e137fa1402eb44c1f50313ff530)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:26.242022 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de7074-e5d4-4d10-bfbc-dc1992ae6399 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:26.290548 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:26.293554 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-91de7074-e5d4-4d10-bfbc-dc1992ae6399 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '747b4e137fa1402eb44c1f50313ff530', 'request_id': 'req-91de7074-e5d4-4d10-bfbc-dc1992ae6399'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:26.302100 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-91de7074-e5d4-4d10-bfbc-dc1992ae6399 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '747b4e137fa1402eb44c1f50313ff530', 'request_id': 'req-91de7074-e5d4-4d10-bfbc-dc1992ae6399'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:26.311875 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-91de7074-e5d4-4d10-bfbc-dc1992ae6399 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '747b4e137fa1402eb44c1f50313ff530', 'request_id': 'req-91de7074-e5d4-4d10-bfbc-dc1992ae6399'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:26.323585 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-91de7074-e5d4-4d10-bfbc-dc1992ae6399 admin admin] The token cache is being invalidate because project 747b4e137fa1402eb44c1f50313ff530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:01:26.323900 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-91de7074-e5d4-4d10-bfbc-dc1992ae6399 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:26.324694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-91de7074-e5d4-4d10-bfbc-dc1992ae6399 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:26.326212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1421/2865] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:01:26 2022] DELETE /identity/v3/projects/747b4e137fa1402eb44c1f50313ff530 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:26.332267 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5246df9-709d-4575-b4a3-2b198d9b733a None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:26.332764 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5246df9-709d-4575-b4a3-2b198d9b733a None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:26.332866 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5246df9-709d-4575-b4a3-2b198d9b733a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:26.353165 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-e5246df9-709d-4575-b4a3-2b198d9b733a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:26.354417 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-e5246df9-709d-4575-b4a3-2b198d9b733a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:26.373671 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:26.421980 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1445/2866] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:26.429966 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4a57f-80e2-43ce-91c6-e3a38eaa7bf0 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:26.430299 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4a57f-80e2-43ce-91c6-e3a38eaa7bf0 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:26.430508 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4a57f-80e2-43ce-91c6-e3a38eaa7bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:26.450729 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-1cc4a57f-80e2-43ce-91c6-e3a38eaa7bf0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:26.452011 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-1cc4a57f-80e2-43ce-91c6-e3a38eaa7bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:26.460079 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1422/2867] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:26.464336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5246df9-709d-4575-b4a3-2b198d9b733a None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:26.475097 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5246df9-709d-4575-b4a3-2b198d9b733a None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:26.479733 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8b7a6-8aec-4115-9eb6-8aada29ca286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:26.480369 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8b7a6-8aec-4115-9eb6-8aada29ca286 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:26.480647 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8b7a6-8aec-4115-9eb6-8aada29ca286 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:26.480917 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8b7a6-8aec-4115-9eb6-8aada29ca286 admin admin] PATH_INFO: `/v3/domains` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:26.482244 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8b7a6-8aec-4115-9eb6-8aada29ca286 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:01:26.482518 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8b7a6-8aec-4115-9eb6-8aada29ca286 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:26.484129 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8b7a6-8aec-4115-9eb6-8aada29ca286 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:26.496262 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1446/2868] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 15:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:01:26.500940 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4a57f-80e2-43ce-91c6-e3a38eaa7bf0 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:26.510515 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4a57f-80e2-43ce-91c6-e3a38eaa7bf0 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:26.515606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc9236-013c-40c2-a186-a89e370bc78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:26.516242 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9236-013c-40c2-a186-a89e370bc78e admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:26.516525 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9236-013c-40c2-a186-a89e370bc78e admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:26.516794 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9236-013c-40c2-a186-a89e370bc78e admin admin] PATH_INFO: `/v3/projects` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:26.517566 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc9236-013c-40c2-a186-a89e370bc78e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:26.519505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc9236-013c-40c2-a186-a89e370bc78e admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:26.552054 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1423/2869] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:01:26 2022] POST /identity/v3/projects => generated 399 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:01:26.556118 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8b7a6-8aec-4115-9eb6-8aada29ca286 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:26.566666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8b7a6-8aec-4115-9eb6-8aada29ca286 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:26.639014 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32eda9-6772-424d-aa4c-a5fddec09802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:26.640184 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32eda9-6772-424d-aa4c-a5fddec09802 admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:26.640494 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32eda9-6772-424d-aa4c-a5fddec09802 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:26.640968 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32eda9-6772-424d-aa4c-a5fddec09802 admin admin] PATH_INFO: `/v3/users` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:26.641628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32eda9-6772-424d-aa4c-a5fddec09802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:26.646009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32eda9-6772-424d-aa4c-a5fddec09802 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:26.677849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1447/2870] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:01:26 2022] POST /identity/v3/users => generated 300 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 19 15:01:26.678748 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc9236-013c-40c2-a186-a89e370bc78e admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:26.699448 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc9236-013c-40c2-a186-a89e370bc78e admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:26.704607 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518618a-6370-4a4c-9278-fccd0cb6897f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:26.705225 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518618a-6370-4a4c-9278-fccd0cb6897f admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:26.705503 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518618a-6370-4a4c-9278-fccd0cb6897f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:26.705775 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c518618a-6370-4a4c-9278-fccd0cb6897f admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:26.706484 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518618a-6370-4a4c-9278-fccd0cb6897f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:26.708277 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c518618a-6370-4a4c-9278-fccd0cb6897f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:26.719355 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1424/2871] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:01:26.723516 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32eda9-6772-424d-aa4c-a5fddec09802 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:26.737902 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32eda9-6772-424d-aa4c-a5fddec09802 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:26.742651 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84deb3-a8b3-413e-8949-8f7830e5081e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:26.743292 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84deb3-a8b3-413e-8949-8f7830e5081e admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:26.743713 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84deb3-a8b3-413e-8949-8f7830e5081e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:26.743918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84deb3-a8b3-413e-8949-8f7830e5081e admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:26.744990 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84deb3-a8b3-413e-8949-8f7830e5081e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:26.746360 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84deb3-a8b3-413e-8949-8f7830e5081e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:26.760615 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1448/2872] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:01:26.765614 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518618a-6370-4a4c-9278-fccd0cb6897f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:26.776720 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c518618a-6370-4a4c-9278-fccd0cb6897f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:26.784885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd4fb7-50bc-4a05-bf3c-b6ce5bc0e1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:26.785965 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd4fb7-50bc-4a05-bf3c-b6ce5bc0e1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:26.786191 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd4fb7-50bc-4a05-bf3c-b6ce5bc0e1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:26.786469 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd4fb7-50bc-4a05-bf3c-b6ce5bc0e1b3 admin admin] PATH_INFO: `/v3/projects/8890c2f205764683be6b1468526aab7a/users/44bdd1b0182a47e498b98c66c707f0d8/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:26.802819 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd4fb7-50bc-4a05-bf3c-b6ce5bc0e1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8890c2f205764683be6b1468526aab7a, user_id=44bdd1b0182a47e498b98c66c707f0d8, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:26.805748 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd4fb7-50bc-4a05-bf3c-b6ce5bc0e1b3 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:26.822883 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1425/2873] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:01:26 2022] PUT /identity/v3/projects/8890c2f205764683be6b1468526aab7a/users/44bdd1b0182a47e498b98c66c707f0d8/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:26.893431 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63d185-df6d-482e-8070-87aa898d97ef None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:26.893723 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63d185-df6d-482e-8070-87aa898d97ef None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:26.894005 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63d185-df6d-482e-8070-87aa898d97ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:26.925535 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1c63d185-df6d-482e-8070-87aa898d97ef None None] MFA Rules not processed for user `44bdd1b0182a47e498b98c66c707f0d8`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:26.926748 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1c63d185-df6d-482e-8070-87aa898d97ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:27.006383 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1449/2874] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:27.018923 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd4fb7-50bc-4a05-bf3c-b6ce5bc0e1b3 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:27.020747 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ffcd4fb7-50bc-4a05-bf3c-b6ce5bc0e1b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:27.090109 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd4fb7-50bc-4a05-bf3c-b6ce5bc0e1b3 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:27.094439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ca0d6-2e48-46bb-ba6e-fde60580a5e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '006fe9ba3b0a40f29fa2be7620e548c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:27.095071 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ca0d6-2e48-46bb-ba6e-fde60580a5e6 service cinder] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:27.095355 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ca0d6-2e48-46bb-ba6e-fde60580a5e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:27.095652 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ca0d6-2e48-46bb-ba6e-fde60580a5e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:27.105117 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ca0d6-2e48-46bb-ba6e-fde60580a5e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:27.106591 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ca0d6-2e48-46bb-ba6e-fde60580a5e6 service cinder] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:27.117743 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1426/2875] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:01:27 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:01:27.401287 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63d185-df6d-482e-8070-87aa898d97ef None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:27.402736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1c63d185-df6d-482e-8070-87aa898d97ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:27.428495 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:27.542527 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63d185-df6d-482e-8070-87aa898d97ef None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:27.549682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19142c57-2d5b-44e2-a81f-fe06dbc6714c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:27.550506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19142c57-2d5b-44e2-a81f-fe06dbc6714c service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:27.550963 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19142c57-2d5b-44e2-a81f-fe06dbc6714c service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:27.551395 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19142c57-2d5b-44e2-a81f-fe06dbc6714c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:27.553548 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-19142c57-2d5b-44e2-a81f-fe06dbc6714c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:27.571736 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19142c57-2d5b-44e2-a81f-fe06dbc6714c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:27.573618 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19142c57-2d5b-44e2-a81f-fe06dbc6714c service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:27.586926 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1450/2876] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:01:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:01:27.717212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ca0d6-2e48-46bb-ba6e-fde60580a5e6 service cinder] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:27.718805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-d71ca0d6-2e48-46bb-ba6e-fde60580a5e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:27.796688 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ca0d6-2e48-46bb-ba6e-fde60580a5e6 service cinder] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:27.804856 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49df7c-3074-452f-bb22-cc8b4c50bb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:27.805994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49df7c-3074-452f-bb22-cc8b4c50bb31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:27.806476 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49df7c-3074-452f-bb22-cc8b4c50bb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:27.807677 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49df7c-3074-452f-bb22-cc8b4c50bb31 admin admin] PATH_INFO: `/v3/users/18bd94a24883495b9fbe066ddc2a953d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:27.809203 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49df7c-3074-452f-bb22-cc8b4c50bb31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18bd94a24883495b9fbe066ddc2a953d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:27.812426 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49df7c-3074-452f-bb22-cc8b4c50bb31 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:27.828417 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:27.880631 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dd49df7c-3074-452f-bb22-cc8b4c50bb31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18bd94a24883495b9fbe066ddc2a953d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:27.891665 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dd49df7c-3074-452f-bb22-cc8b4c50bb31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18bd94a24883495b9fbe066ddc2a953d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:27.909343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dd49df7c-3074-452f-bb22-cc8b4c50bb31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18bd94a24883495b9fbe066ddc2a953d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:27.919789 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dd49df7c-3074-452f-bb22-cc8b4c50bb31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18bd94a24883495b9fbe066ddc2a953d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:27.920565 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-dd49df7c-3074-452f-bb22-cc8b4c50bb31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18bd94a24883495b9fbe066ddc2a953d'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:27.927865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1427/2877] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:01:27 2022] DELETE /identity/v3/users/18bd94a24883495b9fbe066ddc2a953d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:27.933522 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d4d2e-67f5-4d8c-9e64-e6b3dff13a66 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:27.934156 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d4d2e-67f5-4d8c-9e64-e6b3dff13a66 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:27.935405 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d4d2e-67f5-4d8c-9e64-e6b3dff13a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:27.960728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d70d4d2e-67f5-4d8c-9e64-e6b3dff13a66 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:27.962373 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d70d4d2e-67f5-4d8c-9e64-e6b3dff13a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:28.035142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1451/2878] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:28.045305 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49df7c-3074-452f-bb22-cc8b4c50bb31 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:28.047606 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-dd49df7c-3074-452f-bb22-cc8b4c50bb31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:28.113638 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49df7c-3074-452f-bb22-cc8b4c50bb31 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:28.118694 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43400f5-fc3f-4ffa-bc8b-a6c6ed486b45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:28.119434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43400f5-fc3f-4ffa-bc8b-a6c6ed486b45 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:28.120295 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43400f5-fc3f-4ffa-bc8b-a6c6ed486b45 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:28.120583 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43400f5-fc3f-4ffa-bc8b-a6c6ed486b45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:28.131407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43400f5-fc3f-4ffa-bc8b-a6c6ed486b45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:28.132695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43400f5-fc3f-4ffa-bc8b-a6c6ed486b45 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:28.147627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1428/2879] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:01:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:01:28.480147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d4d2e-67f5-4d8c-9e64-e6b3dff13a66 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:28.482773 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d70d4d2e-67f5-4d8c-9e64-e6b3dff13a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:28.502370 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d4d2e-67f5-4d8c-9e64-e6b3dff13a66 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:28.510826 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d84e7c-4ba1-4e2d-a878-5577fe47805e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:28.511594 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d84e7c-4ba1-4e2d-a878-5577fe47805e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:28.511994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d84e7c-4ba1-4e2d-a878-5577fe47805e admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:28.512827 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d84e7c-4ba1-4e2d-a878-5577fe47805e admin admin] PATH_INFO: `/v3/projects/eb4bd6bb69c04dcf8475595904675e20` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:28.514324 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d84e7c-4ba1-4e2d-a878-5577fe47805e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb4bd6bb69c04dcf8475595904675e20)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:28.516801 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d84e7c-4ba1-4e2d-a878-5577fe47805e admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:28.580275 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-55d84e7c-4ba1-4e2d-a878-5577fe47805e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb4bd6bb69c04dcf8475595904675e20', 'request_id': 'req-55d84e7c-4ba1-4e2d-a878-5577fe47805e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:28.587372 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-55d84e7c-4ba1-4e2d-a878-5577fe47805e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb4bd6bb69c04dcf8475595904675e20', 'request_id': 'req-55d84e7c-4ba1-4e2d-a878-5577fe47805e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:28.595221 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-55d84e7c-4ba1-4e2d-a878-5577fe47805e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb4bd6bb69c04dcf8475595904675e20', 'request_id': 'req-55d84e7c-4ba1-4e2d-a878-5577fe47805e'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:28.609290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-55d84e7c-4ba1-4e2d-a878-5577fe47805e admin admin] The token cache is being invalidate because project eb4bd6bb69c04dcf8475595904675e20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:01:28.609587 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-55d84e7c-4ba1-4e2d-a878-5577fe47805e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:28.610226 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-55d84e7c-4ba1-4e2d-a878-5577fe47805e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:28.611751 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1452/2880] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:01:28 2022] DELETE /identity/v3/projects/eb4bd6bb69c04dcf8475595904675e20 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:37.357274 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43400f5-fc3f-4ffa-bc8b-a6c6ed486b45 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:37.358803 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e43400f5-fc3f-4ffa-bc8b-a6c6ed486b45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:37.373897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43400f5-fc3f-4ffa-bc8b-a6c6ed486b45 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:37.430096 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0032ef-b0a5-4602-b772-3dd52371d4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:37.430718 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0032ef-b0a5-4602-b772-3dd52371d4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:37.430994 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0032ef-b0a5-4602-b772-3dd52371d4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:37.431264 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0032ef-b0a5-4602-b772-3dd52371d4f9 admin admin] PATH_INFO: `/v3/users/44bdd1b0182a47e498b98c66c707f0d8` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:37.432394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0032ef-b0a5-4602-b772-3dd52371d4f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44bdd1b0182a47e498b98c66c707f0d8)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:37.435174 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0032ef-b0a5-4602-b772-3dd52371d4f9 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:37.471659 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:37.481945 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-aa0032ef-b0a5-4602-b772-3dd52371d4f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44bdd1b0182a47e498b98c66c707f0d8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:37.489619 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-aa0032ef-b0a5-4602-b772-3dd52371d4f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44bdd1b0182a47e498b98c66c707f0d8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:37.502069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-aa0032ef-b0a5-4602-b772-3dd52371d4f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44bdd1b0182a47e498b98c66c707f0d8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:37.511086 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-aa0032ef-b0a5-4602-b772-3dd52371d4f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44bdd1b0182a47e498b98c66c707f0d8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:37.511695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-aa0032ef-b0a5-4602-b772-3dd52371d4f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44bdd1b0182a47e498b98c66c707f0d8'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:37.518026 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1429/2881] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:01:37 2022] DELETE /identity/v3/users/44bdd1b0182a47e498b98c66c707f0d8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:37.522572 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9bff3-574a-495b-af25-dd20922bf499 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:37.523069 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9bff3-574a-495b-af25-dd20922bf499 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:37.523178 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9bff3-574a-495b-af25-dd20922bf499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:37.538452 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:37.545681 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-1ef9bff3-574a-495b-af25-dd20922bf499 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:37.547738 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1ef9bff3-574a-495b-af25-dd20922bf499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:37.607787 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1453/2882] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:37.617368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0032ef-b0a5-4602-b772-3dd52371d4f9 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:37.618769 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-aa0032ef-b0a5-4602-b772-3dd52371d4f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:37.676981 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0032ef-b0a5-4602-b772-3dd52371d4f9 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:37.680717 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad299f-44e5-4b9e-af2d-3037e7b65662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:37.681327 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad299f-44e5-4b9e-af2d-3037e7b65662 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:37.681609 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad299f-44e5-4b9e-af2d-3037e7b65662 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:37.681885 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad299f-44e5-4b9e-af2d-3037e7b65662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:37.691642 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad299f-44e5-4b9e-af2d-3037e7b65662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:37.693127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad299f-44e5-4b9e-af2d-3037e7b65662 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:37.703294 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1430/2883] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:01:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:01:37.975576 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9bff3-574a-495b-af25-dd20922bf499 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:37.977040 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-1ef9bff3-574a-495b-af25-dd20922bf499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:37.989586 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9bff3-574a-495b-af25-dd20922bf499 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:37.994240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca83ed-fd3a-477c-8686-b221c57ca888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:37.994875 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca83ed-fd3a-477c-8686-b221c57ca888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:37.995154 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca83ed-fd3a-477c-8686-b221c57ca888 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:37.995426 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca83ed-fd3a-477c-8686-b221c57ca888 admin admin] PATH_INFO: `/v3/projects/8890c2f205764683be6b1468526aab7a` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:37.996614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca83ed-fd3a-477c-8686-b221c57ca888 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8890c2f205764683be6b1468526aab7a)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:37.998517 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca83ed-fd3a-477c-8686-b221c57ca888 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:38.048703 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-94ca83ed-fd3a-477c-8686-b221c57ca888 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8890c2f205764683be6b1468526aab7a', 'request_id': 'req-94ca83ed-fd3a-477c-8686-b221c57ca888'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:38.054020 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-94ca83ed-fd3a-477c-8686-b221c57ca888 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8890c2f205764683be6b1468526aab7a', 'request_id': 'req-94ca83ed-fd3a-477c-8686-b221c57ca888'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:38.060415 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-94ca83ed-fd3a-477c-8686-b221c57ca888 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8890c2f205764683be6b1468526aab7a', 'request_id': 'req-94ca83ed-fd3a-477c-8686-b221c57ca888'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:38.072638 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-94ca83ed-fd3a-477c-8686-b221c57ca888 admin admin] The token cache is being invalidate because project 8890c2f205764683be6b1468526aab7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:01:38.072918 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-94ca83ed-fd3a-477c-8686-b221c57ca888 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:38.073596 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-94ca83ed-fd3a-477c-8686-b221c57ca888 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:38.075706 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1454/2884] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:01:37 2022] DELETE /identity/v3/projects/8890c2f205764683be6b1468526aab7a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:47.492255 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad299f-44e5-4b9e-af2d-3037e7b65662 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:47.493674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-e5ad299f-44e5-4b9e-af2d-3037e7b65662 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:47.506733 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad299f-44e5-4b9e-af2d-3037e7b65662 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:47.562207 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d0d4f4-aa87-4833-a33c-1414d8c8c81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:47.562838 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0d4f4-aa87-4833-a33c-1414d8c8c81d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:47.563118 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0d4f4-aa87-4833-a33c-1414d8c8c81d admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:47.563392 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d0d4f4-aa87-4833-a33c-1414d8c8c81d admin admin] PATH_INFO: `/v3/users/420f20f1b74742dfbe87cee54406600e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:47.564563 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d0d4f4-aa87-4833-a33c-1414d8c8c81d admin admin] RBAC: Authorizing `identity:delete_user(user_id=420f20f1b74742dfbe87cee54406600e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:47.566482 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d0d4f4-aa87-4833-a33c-1414d8c8c81d admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:47.614128 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-94d0d4f4-aa87-4833-a33c-1414d8c8c81d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '420f20f1b74742dfbe87cee54406600e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:47.620897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-94d0d4f4-aa87-4833-a33c-1414d8c8c81d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '420f20f1b74742dfbe87cee54406600e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:47.623206 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:47.634920 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-94d0d4f4-aa87-4833-a33c-1414d8c8c81d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '420f20f1b74742dfbe87cee54406600e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:47.642423 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-94d0d4f4-aa87-4833-a33c-1414d8c8c81d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '420f20f1b74742dfbe87cee54406600e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:47.643173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-94d0d4f4-aa87-4833-a33c-1414d8c8c81d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '420f20f1b74742dfbe87cee54406600e'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:47.649250 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1431/2885] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:01:47 2022] DELETE /identity/v3/users/420f20f1b74742dfbe87cee54406600e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:47.653730 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d48e8-b278-48d1-914d-78844ca354a0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:47.654227 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d48e8-b278-48d1-914d-78844ca354a0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:47.654336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d48e8-b278-48d1-914d-78844ca354a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:47.673350 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-5e9d48e8-b278-48d1-914d-78844ca354a0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:47.675269 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5e9d48e8-b278-48d1-914d-78844ca354a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:47.734989 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1455/2886] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:47.744767 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d0d4f4-aa87-4833-a33c-1414d8c8c81d admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:47.747166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-94d0d4f4-aa87-4833-a33c-1414d8c8c81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:47.804672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d0d4f4-aa87-4833-a33c-1414d8c8c81d admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:47.809247 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a14b4-89d5-4e08-9d0c-688f9a467e0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:47.809846 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a14b4-89d5-4e08-9d0c-688f9a467e0e service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:47.810122 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a14b4-89d5-4e08-9d0c-688f9a467e0e service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:47.810394 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a14b4-89d5-4e08-9d0c-688f9a467e0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:47.819150 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a14b4-89d5-4e08-9d0c-688f9a467e0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:47.820596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a14b4-89d5-4e08-9d0c-688f9a467e0e service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:47.829844 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1432/2887] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:01:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:01:48.108545 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d48e8-b278-48d1-914d-78844ca354a0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:48.109994 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5e9d48e8-b278-48d1-914d-78844ca354a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:48.122803 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d48e8-b278-48d1-914d-78844ca354a0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:48.127628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d8a7d-f817-4741-a822-e11d3ff7520a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:48.128298 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d8a7d-f817-4741-a822-e11d3ff7520a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:48.128589 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d8a7d-f817-4741-a822-e11d3ff7520a admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:48.128864 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d8a7d-f817-4741-a822-e11d3ff7520a admin admin] PATH_INFO: `/v3/projects/3989c7cfaaaf46be8ccb361ba23a17e2` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:48.130058 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d8a7d-f817-4741-a822-e11d3ff7520a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3989c7cfaaaf46be8ccb361ba23a17e2)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:48.132009 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d8a7d-f817-4741-a822-e11d3ff7520a admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:48.163750 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:48.183433 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1a4d8a7d-f817-4741-a822-e11d3ff7520a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3989c7cfaaaf46be8ccb361ba23a17e2', 'request_id': 'req-1a4d8a7d-f817-4741-a822-e11d3ff7520a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:48.189100 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1a4d8a7d-f817-4741-a822-e11d3ff7520a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3989c7cfaaaf46be8ccb361ba23a17e2', 'request_id': 'req-1a4d8a7d-f817-4741-a822-e11d3ff7520a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:48.195603 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1a4d8a7d-f817-4741-a822-e11d3ff7520a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3989c7cfaaaf46be8ccb361ba23a17e2', 'request_id': 'req-1a4d8a7d-f817-4741-a822-e11d3ff7520a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:48.208818 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1a4d8a7d-f817-4741-a822-e11d3ff7520a admin admin] The token cache is being invalidate because project 3989c7cfaaaf46be8ccb361ba23a17e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:01:48.209232 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1a4d8a7d-f817-4741-a822-e11d3ff7520a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:48.209958 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-1a4d8a7d-f817-4741-a822-e11d3ff7520a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:01:48.211602 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1456/2888] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:01:48 2022] DELETE /identity/v3/projects/3989c7cfaaaf46be8ccb361ba23a17e2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:48.217286 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4602cca-42bb-4eb7-a7ca-d4dde8b66e5e None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:48.217574 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4602cca-42bb-4eb7-a7ca-d4dde8b66e5e None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:48.217848 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4602cca-42bb-4eb7-a7ca-d4dde8b66e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:48.238038 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-b4602cca-42bb-4eb7-a7ca-d4dde8b66e5e None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:48.239212 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b4602cca-42bb-4eb7-a7ca-d4dde8b66e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:48.302343 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1433/2889] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:48.311030 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ba7fa-3a42-422d-bafc-2e286d0296b0 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:48.311484 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ba7fa-3a42-422d-bafc-2e286d0296b0 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:48.311922 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ba7fa-3a42-422d-bafc-2e286d0296b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:48.332078 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-a74ba7fa-3a42-422d-bafc-2e286d0296b0 None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:48.341330 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a74ba7fa-3a42-422d-bafc-2e286d0296b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:48.351567 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1457/2890] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:48.356724 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4602cca-42bb-4eb7-a7ca-d4dde8b66e5e None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:48.366975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4602cca-42bb-4eb7-a7ca-d4dde8b66e5e None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:48.371690 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2d4d7-87ab-4292-af52-3f491d200c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:48.372321 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2d4d7-87ab-4292-af52-3f491d200c12 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:48.372605 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2d4d7-87ab-4292-af52-3f491d200c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:48.372875 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2d4d7-87ab-4292-af52-3f491d200c12 admin admin] PATH_INFO: `/v3/domains` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:48.373507 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2d4d7-87ab-4292-af52-3f491d200c12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 19 15:01:48.373808 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2d4d7-87ab-4292-af52-3f491d200c12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:48.375407 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2d4d7-87ab-4292-af52-3f491d200c12 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:48.387019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1434/2891] 10.222.0.60 () {58 vars in 1153 bytes} [Mon Sep 19 15:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 19 15:01:48.391362 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ba7fa-3a42-422d-bafc-2e286d0296b0 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:48.400726 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ba7fa-3a42-422d-bafc-2e286d0296b0 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:48.406049 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5d102-458e-4f44-9933-4ecaed8131cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:48.406648 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5d102-458e-4f44-9933-4ecaed8131cb admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:48.406934 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5d102-458e-4f44-9933-4ecaed8131cb admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:48.407229 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5d102-458e-4f44-9933-4ecaed8131cb admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:48.407876 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5d102-458e-4f44-9933-4ecaed8131cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:48.409782 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5d102-458e-4f44-9933-4ecaed8131cb admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:48.437430 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1458/2892] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:01:48 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:01:48.441735 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2d4d7-87ab-4292-af52-3f491d200c12 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:48.451943 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2d4d7-87ab-4292-af52-3f491d200c12 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:48.510280 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30047a0-b9d0-4a3b-a0fa-3d6817da881f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:48.510930 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30047a0-b9d0-4a3b-a0fa-3d6817da881f admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:48.511094 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30047a0-b9d0-4a3b-a0fa-3d6817da881f admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:48.511364 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30047a0-b9d0-4a3b-a0fa-3d6817da881f admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:48.512674 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30047a0-b9d0-4a3b-a0fa-3d6817da881f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:48.514612 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30047a0-b9d0-4a3b-a0fa-3d6817da881f admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:48.525466 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:48.537573 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1435/2893] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:01:48 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:01:48.541683 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5d102-458e-4f44-9933-4ecaed8131cb admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:48.552699 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5d102-458e-4f44-9933-4ecaed8131cb admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:48.557388 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779a702-6283-44b5-97da-4fccabed23fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:48.558239 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779a702-6283-44b5-97da-4fccabed23fb admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:48.559147 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779a702-6283-44b5-97da-4fccabed23fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:48.559423 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779a702-6283-44b5-97da-4fccabed23fb admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:48.560092 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779a702-6283-44b5-97da-4fccabed23fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:48.561676 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779a702-6283-44b5-97da-4fccabed23fb admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:48.571723 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1459/2894] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:01:48.575748 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30047a0-b9d0-4a3b-a0fa-3d6817da881f admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:48.585715 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30047a0-b9d0-4a3b-a0fa-3d6817da881f admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:48.590505 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b469d2c-8c0b-4e10-b2e1-42df52cb7c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:48.591177 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b469d2c-8c0b-4e10-b2e1-42df52cb7c83 admin admin] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:48.591457 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b469d2c-8c0b-4e10-b2e1-42df52cb7c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:48.591727 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b469d2c-8c0b-4e10-b2e1-42df52cb7c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:48.592399 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b469d2c-8c0b-4e10-b2e1-42df52cb7c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:48.593987 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b469d2c-8c0b-4e10-b2e1-42df52cb7c83 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:48.604924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1436/2895] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:01:48.609707 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779a702-6283-44b5-97da-4fccabed23fb admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:48.619592 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779a702-6283-44b5-97da-4fccabed23fb admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:48.624897 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8dc29-d380-410f-b070-c81d0cdd6da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:48.625566 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8dc29-d380-410f-b070-c81d0cdd6da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:48.625849 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8dc29-d380-410f-b070-c81d0cdd6da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:48.626146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8dc29-d380-410f-b070-c81d0cdd6da2 admin admin] PATH_INFO: `/v3/projects/5fca221e257f44dea6b74f7737978739/users/a3d7fd2980ce48a49931642b01bab212/roles/0a6592b9a0b64df6938aba0a010c2256` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:48.640518 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8dc29-d380-410f-b070-c81d0cdd6da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fca221e257f44dea6b74f7737978739, user_id=a3d7fd2980ce48a49931642b01bab212, role_id=0a6592b9a0b64df6938aba0a010c2256)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:48.643146 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8dc29-d380-410f-b070-c81d0cdd6da2 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:48.655788 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1460/2896] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:01:48 2022] PUT /identity/v3/projects/5fca221e257f44dea6b74f7737978739/users/a3d7fd2980ce48a49931642b01bab212/roles/0a6592b9a0b64df6938aba0a010c2256 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 15:01:48.660921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1e114-c1da-4dfc-9b11-54b523c8c99a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:48.661226 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1e114-c1da-4dfc-9b11-54b523c8c99a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:48.661502 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1e114-c1da-4dfc-9b11-54b523c8c99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:48.680510 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-07c1e114-c1da-4dfc-9b11-54b523c8c99a None None] MFA Rules not processed for user `7d770189a7bb4b81b8a42702f35fc84d`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:48.681695 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-07c1e114-c1da-4dfc-9b11-54b523c8c99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:48.740033 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1437/2897] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:48.750672 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8dc29-d380-410f-b070-c81d0cdd6da2 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:48.752411 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-5bd8dc29-d380-410f-b070-c81d0cdd6da2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:48.812434 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8dc29-d380-410f-b070-c81d0cdd6da2 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:48.816240 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d0721-f9c0-44bc-ad35-63ad9470d07b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:48.816859 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d0721-f9c0-44bc-ad35-63ad9470d07b service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:48.817138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d0721-f9c0-44bc-ad35-63ad9470d07b service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:48.817415 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d0721-f9c0-44bc-ad35-63ad9470d07b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:48.827404 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d0721-f9c0-44bc-ad35-63ad9470d07b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:48.828936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d0721-f9c0-44bc-ad35-63ad9470d07b service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:48.838519 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1461/2898] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:01:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:01:52.697358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed986050-ebf0-4eba-a603-d870b1cb281a None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:52.697865 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed986050-ebf0-4eba-a603-d870b1cb281a None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:52.697992 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed986050-ebf0-4eba-a603-d870b1cb281a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:52.728799 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-ed986050-ebf0-4eba-a603-d870b1cb281a None None] MFA Rules not processed for user `a3d7fd2980ce48a49931642b01bab212`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:52.730196 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-ed986050-ebf0-4eba-a603-d870b1cb281a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:52.799135 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1438/2899] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:52.803743 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d0721-f9c0-44bc-ad35-63ad9470d07b service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:52.814890 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d0721-f9c0-44bc-ad35-63ad9470d07b service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:52.819745 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae87ab-f417-42aa-899f-44eac2fb86df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:52.820415 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae87ab-f417-42aa-899f-44eac2fb86df admin admin] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:52.820700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae87ab-f417-42aa-899f-44eac2fb86df admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:52.820978 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae87ab-f417-42aa-899f-44eac2fb86df admin admin] PATH_INFO: `/v3/projects` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:52.821611 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae87ab-f417-42aa-899f-44eac2fb86df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:52.823522 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae87ab-f417-42aa-899f-44eac2fb86df admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:52.855416 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1462/2900] 10.222.0.60 () {60 vars in 1153 bytes} [Mon Sep 19 15:01:52 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:01:52.860173 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed986050-ebf0-4eba-a603-d870b1cb281a None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:52.875166 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed986050-ebf0-4eba-a603-d870b1cb281a None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:52.936530 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf130756-e038-42d8-83d2-ce4cccfd9f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:52.937309 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf130756-e038-42d8-83d2-ce4cccfd9f12 admin admin] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:52.937623 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf130756-e038-42d8-83d2-ce4cccfd9f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:52.937901 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf130756-e038-42d8-83d2-ce4cccfd9f12 admin admin] PATH_INFO: `/v3/users` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:52.938545 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf130756-e038-42d8-83d2-ce4cccfd9f12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:52.940611 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf130756-e038-42d8-83d2-ce4cccfd9f12 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:52.965974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1439/2901] 10.222.0.60 () {60 vars in 1144 bytes} [Mon Sep 19 15:01:52 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 19 15:01:52.970088 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae87ab-f417-42aa-899f-44eac2fb86df admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:52.980629 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae87ab-f417-42aa-899f-44eac2fb86df admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:52.985438 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309a868-66eb-49f2-8056-503c1bfade78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:52.986065 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309a868-66eb-49f2-8056-503c1bfade78 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:52.986380 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309a868-66eb-49f2-8056-503c1bfade78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:52.986660 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309a868-66eb-49f2-8056-503c1bfade78 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:52.987305 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c309a868-66eb-49f2-8056-503c1bfade78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:52.989201 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c309a868-66eb-49f2-8056-503c1bfade78 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:53.001957 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1463/2902] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 19 15:01:53.006155 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf130756-e038-42d8-83d2-ce4cccfd9f12 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:53.013359 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:53.017429 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf130756-e038-42d8-83d2-ce4cccfd9f12 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:53.023921 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1366579-4ae8-4235-a0a1-35e57a718540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:53.024596 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1366579-4ae8-4235-a0a1-35e57a718540 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:53.024876 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1366579-4ae8-4235-a0a1-35e57a718540 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:53.025151 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1366579-4ae8-4235-a0a1-35e57a718540 admin admin] PATH_INFO: `/v3/projects/677fa01f1558491dac05c307944f365d/users/361efb27f3ae421fb721931ab0318d84/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:53.041411 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1366579-4ae8-4235-a0a1-35e57a718540 admin admin] RBAC: Authorizing `identity:create_grant(project_id=677fa01f1558491dac05c307944f365d, user_id=361efb27f3ae421fb721931ab0318d84, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:53.044815 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1366579-4ae8-4235-a0a1-35e57a718540 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:53.059906 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1440/2903] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:01:53 2022] PUT /identity/v3/projects/677fa01f1558491dac05c307944f365d/users/361efb27f3ae421fb721931ab0318d84/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:53.064207 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309a868-66eb-49f2-8056-503c1bfade78 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:53.075357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309a868-66eb-49f2-8056-503c1bfade78 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:53.094230 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:01:53.138913 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739b0c1-97d6-413d-a4cd-3ded1df5ae35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:53.139623 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739b0c1-97d6-413d-a4cd-3ded1df5ae35 admin admin] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:53.140094 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739b0c1-97d6-413d-a4cd-3ded1df5ae35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:53.140440 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739b0c1-97d6-413d-a4cd-3ded1df5ae35 admin admin] PATH_INFO: `/v3/roles` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:53.141163 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739b0c1-97d6-413d-a4cd-3ded1df5ae35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:53.142844 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739b0c1-97d6-413d-a4cd-3ded1df5ae35 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:53.155616 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1464/2904] 10.222.0.60 () {58 vars in 1122 bytes} [Mon Sep 19 15:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 19 15:01:53.159791 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1366579-4ae8-4235-a0a1-35e57a718540 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:53.174035 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1366579-4ae8-4235-a0a1-35e57a718540 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:53.186164 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58244f74-d9bb-487b-9bb3-799f737857e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:53.186976 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58244f74-d9bb-487b-9bb3-799f737857e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:53.187348 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58244f74-d9bb-487b-9bb3-799f737857e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:53.187722 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58244f74-d9bb-487b-9bb3-799f737857e1 admin admin] PATH_INFO: `/v3/projects/677fa01f1558491dac05c307944f365d/users/361efb27f3ae421fb721931ab0318d84/roles/d3ae59ea5d7f4bcc964c8ff473925f8e` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:53.190703 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58244f74-d9bb-487b-9bb3-799f737857e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=677fa01f1558491dac05c307944f365d, user_id=361efb27f3ae421fb721931ab0318d84, role_id=d3ae59ea5d7f4bcc964c8ff473925f8e)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:53.195061 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58244f74-d9bb-487b-9bb3-799f737857e1 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:53.206074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1441/2905] 10.222.0.60 () {60 vars in 1483 bytes} [Mon Sep 19 15:01:53 2022] PUT /identity/v3/projects/677fa01f1558491dac05c307944f365d/users/361efb27f3ae421fb721931ab0318d84/roles/d3ae59ea5d7f4bcc964c8ff473925f8e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:01:57.722424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c849d-c0fe-48c7-9c12-f908ea602418 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:57.722942 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c849d-c0fe-48c7-9c12-f908ea602418 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:57.723063 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c849d-c0fe-48c7-9c12-f908ea602418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:57.756424 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-842c849d-c0fe-48c7-9c12-f908ea602418 None None] MFA Rules not processed for user `361efb27f3ae421fb721931ab0318d84`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:57.757655 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-842c849d-c0fe-48c7-9c12-f908ea602418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:57.850970 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1465/2906] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:01:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:57.863924 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58244f74-d9bb-487b-9bb3-799f737857e1 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:57.866797 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-58244f74-d9bb-487b-9bb3-799f737857e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:57.942386 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58244f74-d9bb-487b-9bb3-799f737857e1 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:57.946069 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19b761-a905-4ac4-8a72-25148db7f063 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'caab7aaa72634b358d717768fbcb48d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:57.946705 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19b761-a905-4ac4-8a72-25148db7f063 service glance] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:57.946980 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19b761-a905-4ac4-8a72-25148db7f063 service glance] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:57.947248 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19b761-a905-4ac4-8a72-25148db7f063 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:58.010969 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f19b761-a905-4ac4-8a72-25148db7f063 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:58.012943 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f19b761-a905-4ac4-8a72-25148db7f063 service glance] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:58.022486 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1442/2907] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:01:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:01:58.169787 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275ec44-e39c-46b2-a367-98dd32e1de98 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:58.170317 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275ec44-e39c-46b2-a367-98dd32e1de98 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:58.170666 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275ec44-e39c-46b2-a367-98dd32e1de98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:58.191157 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-d275ec44-e39c-46b2-a367-98dd32e1de98 None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:01:58.192413 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-d275ec44-e39c-46b2-a367-98dd32e1de98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:58.252680 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1466/2908] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Sep 19 15:01:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:01:58.306464 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19b761-a905-4ac4-8a72-25148db7f063 service glance] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:01:58.308329 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-9f19b761-a905-4ac4-8a72-25148db7f063 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:01:58.380246 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f19b761-a905-4ac4-8a72-25148db7f063 service glance] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:01:58.388139 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97567adc-2a73-4441-8959-2b8dd4f67bc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:01:58.389095 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97567adc-2a73-4441-8959-2b8dd4f67bc6 service swift] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:01:58.389538 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97567adc-2a73-4441-8959-2b8dd4f67bc6 service swift] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:01:58.390003 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97567adc-2a73-4441-8959-2b8dd4f67bc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:01:58.401358 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97567adc-2a73-4441-8959-2b8dd4f67bc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:01:58.403709 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97567adc-2a73-4441-8959-2b8dd4f67bc6 service swift] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:01:58.413892 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1443/2909] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 15:01:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 15:02:07.175313 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79df13b-d3ff-4a33-bf97-47d21baeeb6d None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:02:07.176159 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79df13b-d3ff-4a33-bf97-47d21baeeb6d None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:02:07.176437 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79df13b-d3ff-4a33-bf97-47d21baeeb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:02:07.195573 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-b79df13b-d3ff-4a33-bf97-47d21baeeb6d None None] MFA Rules not processed for user `a3d7fd2980ce48a49931642b01bab212`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:02:07.196808 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-b79df13b-d3ff-4a33-bf97-47d21baeeb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:02:07.203634 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1467/2910] 10.222.0.60 () {58 vars in 958 bytes} [Mon Sep 19 15:02:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:02:08.409115 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=93,keystone:INSERT=2 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:02:09.081939 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510029e-098b-447f-b10e-2514df6ed126 None None] REQUEST_METHOD: `POST` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:02:09.082160 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510029e-098b-447f-b10e-2514df6ed126 None None] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:02:09.082439 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510029e-098b-447f-b10e-2514df6ed126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:02:09.102201 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.auth.core [None req-c510029e-098b-447f-b10e-2514df6ed126 None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:02:09.103406 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c510029e-098b-447f-b10e-2514df6ed126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:02:09.111383 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1444/2911] 10.222.0.60 () {58 vars in 1009 bytes} [Mon Sep 19 15:02:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:02:09.125084 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79df13b-d3ff-4a33-bf97-47d21baeeb6d None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:02:09.137221 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79df13b-d3ff-4a33-bf97-47d21baeeb6d None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:02:09.142205 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b9339-6a8d-4101-af5b-0b8e1eb0d46d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:02:09.142822 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b9339-6a8d-4101-af5b-0b8e1eb0d46d service swift] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:02:09.143113 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b9339-6a8d-4101-af5b-0b8e1eb0d46d service swift] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:02:09.143394 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b9339-6a8d-4101-af5b-0b8e1eb0d46d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:02:09.152558 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b9339-6a8d-4101-af5b-0b8e1eb0d46d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:02:09.154174 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b9339-6a8d-4101-af5b-0b8e1eb0d46d service swift] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:02:09.162506 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1468/2912] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 15:02:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 15:02:19.096532 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=3 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:02:19.159188 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=88 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:02:21.056153 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510029e-098b-447f-b10e-2514df6ed126 None None] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:02:21.057881 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-c510029e-098b-447f-b10e-2514df6ed126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:02:21.122645 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510029e-098b-447f-b10e-2514df6ed126 None None] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:02:21.124727 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4360c-3abe-477a-9ce9-e97c9863ec3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:02:21.125344 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4360c-3abe-477a-9ce9-e97c9863ec3a service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:02:21.125627 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4360c-3abe-477a-9ce9-e97c9863ec3a service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:02:21.125947 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4360c-3abe-477a-9ce9-e97c9863ec3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:02:21.127672 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-22c4360c-3abe-477a-9ce9-e97c9863ec3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:02:21.199342 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4360c-3abe-477a-9ce9-e97c9863ec3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:02:21.201480 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4360c-3abe-477a-9ce9-e97c9863ec3a service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:02:21.212485 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1445/2913] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:02:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:02:31.207456 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=54 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:02:41.727367 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b9339-6a8d-4101-af5b-0b8e1eb0d46d service swift] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:02:41.738064 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b9339-6a8d-4101-af5b-0b8e1eb0d46d service swift] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:02:41.741702 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3786ed2-5b02-434b-9c8b-a63bd8616816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:02:41.742297 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3786ed2-5b02-434b-9c8b-a63bd8616816 service neutron] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:02:41.742637 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3786ed2-5b02-434b-9c8b-a63bd8616816 service neutron] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:02:41.742835 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3786ed2-5b02-434b-9c8b-a63bd8616816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:02:41.752253 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3786ed2-5b02-434b-9c8b-a63bd8616816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:02:41.753960 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3786ed2-5b02-434b-9c8b-a63bd8616816 service neutron] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:02:41.764553 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1469/2914] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:02:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:02:51.759467 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=6 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:02:59.294997 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4360c-3abe-477a-9ce9-e97c9863ec3a service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:02:59.304598 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4360c-3abe-477a-9ce9-e97c9863ec3a service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:02:59.309294 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d04314-8dfa-405f-a93d-42519e371503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:02:59.310049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d04314-8dfa-405f-a93d-42519e371503 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:02:59.310913 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d04314-8dfa-405f-a93d-42519e371503 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:02:59.310913 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d04314-8dfa-405f-a93d-42519e371503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:02:59.312805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-a5d04314-8dfa-405f-a93d-42519e371503 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:02:59.323787 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d04314-8dfa-405f-a93d-42519e371503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:02:59.325326 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d04314-8dfa-405f-a93d-42519e371503 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:02:59.334419 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1446/2915] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:02:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:03:08.107256 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3786ed2-5b02-434b-9c8b-a63bd8616816 service neutron] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:03:08.109089 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-f3786ed2-5b02-434b-9c8b-a63bd8616816 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:03:08.126580 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3786ed2-5b02-434b-9c8b-a63bd8616816 service neutron] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:03:08.132172 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5236ce-7425-497d-bf9f-a0a27315d428 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59b5d2fd362b417c9b9be75f88a667ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:03:08.132774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5236ce-7425-497d-bf9f-a0a27315d428 service nova] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:03:08.133049 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5236ce-7425-497d-bf9f-a0a27315d428 service nova] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:03:08.133319 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5236ce-7425-497d-bf9f-a0a27315d428 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:03:08.135031 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9f5236ce-7425-497d-bf9f-a0a27315d428 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:03:08.147533 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5236ce-7425-497d-bf9f-a0a27315d428 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:03:08.149138 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5236ce-7425-497d-bf9f-a0a27315d428 service nova] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:03:08.159556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1470/2916] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:03:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:03:09.330521 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=6 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:03:11.206110 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d04314-8dfa-405f-a93d-42519e371503 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:03:11.215529 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d04314-8dfa-405f-a93d-42519e371503 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:03:11.219168 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7781699-e15f-40ad-8408-47b48998c802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbef7b24f134f5eb1193aed1b7ee69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:03:11.219760 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7781699-e15f-40ad-8408-47b48998c802 service neutron] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:03:11.220074 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7781699-e15f-40ad-8408-47b48998c802 service neutron] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:03:11.220342 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7781699-e15f-40ad-8408-47b48998c802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:03:11.222065 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-b7781699-e15f-40ad-8408-47b48998c802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:03:11.286076 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7781699-e15f-40ad-8408-47b48998c802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:03:11.287521 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7781699-e15f-40ad-8408-47b48998c802 service neutron] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:03:11.298019 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1447/2917] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:03:14.211617 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5236ce-7425-497d-bf9f-a0a27315d428 service nova] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:03:14.213157 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-9f5236ce-7425-497d-bf9f-a0a27315d428 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:03:14.307413 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5236ce-7425-497d-bf9f-a0a27315d428 service nova] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:03:14.313177 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661e57b-ae1d-47f1-b75e-60df3c283b50 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '568189358b154b66a729fae2f0411d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:03:14.313938 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661e57b-ae1d-47f1-b75e-60df3c283b50 service placement] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:03:14.314402 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661e57b-ae1d-47f1-b75e-60df3c283b50 service placement] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:03:14.314684 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661e57b-ae1d-47f1-b75e-60df3c283b50 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:03:14.316476 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3661e57b-ae1d-47f1-b75e-60df3c283b50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:03:14.330774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661e57b-ae1d-47f1-b75e-60df3c283b50 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:03:14.332333 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3661e57b-ae1d-47f1-b75e-60df3c283b50 service placement] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:03:14.344357 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1471/2918] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:03:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:03:17.411594 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7781699-e15f-40ad-8408-47b48998c802 service neutron] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:03:17.423870 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7781699-e15f-40ad-8408-47b48998c802 service neutron] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:03:17.430454 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130c7e1-6c12-47b7-af6e-0da0cf107d32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '568189358b154b66a729fae2f0411d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:03:17.431087 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130c7e1-6c12-47b7-af6e-0da0cf107d32 service placement] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:03:17.431369 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130c7e1-6c12-47b7-af6e-0da0cf107d32 service placement] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:03:17.431644 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f130c7e1-6c12-47b7-af6e-0da0cf107d32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:03:17.434199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-f130c7e1-6c12-47b7-af6e-0da0cf107d32 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:03:17.449805 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130c7e1-6c12-47b7-af6e-0da0cf107d32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:03:17.451975 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f130c7e1-6c12-47b7-af6e-0da0cf107d32 service placement] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:03:17.464974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1448/2919] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:03:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:03:17.496642 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661e57b-ae1d-47f1-b75e-60df3c283b50 service placement] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:03:17.497768 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3661e57b-ae1d-47f1-b75e-60df3c283b50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:03:17.512745 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661e57b-ae1d-47f1-b75e-60df3c283b50 service placement] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:03:17.517600 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce591e-ae55-4b86-a4af-e3b52d40bfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:03:17.518235 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce591e-ae55-4b86-a4af-e3b52d40bfd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:03:17.518516 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce591e-ae55-4b86-a4af-e3b52d40bfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:03:17.518843 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce591e-ae55-4b86-a4af-e3b52d40bfd5 admin admin] PATH_INFO: `/v3/users/52173f24f42a481380b047abe76bca57` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:03:17.520029 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce591e-ae55-4b86-a4af-e3b52d40bfd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52173f24f42a481380b047abe76bca57)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:03:17.521936 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce591e-ae55-4b86-a4af-e3b52d40bfd5 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:03:17.576042 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ebce591e-ae55-4b86-a4af-e3b52d40bfd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52173f24f42a481380b047abe76bca57'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:17.583377 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ebce591e-ae55-4b86-a4af-e3b52d40bfd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52173f24f42a481380b047abe76bca57'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:17.596390 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ebce591e-ae55-4b86-a4af-e3b52d40bfd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52173f24f42a481380b047abe76bca57'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:17.606339 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ebce591e-ae55-4b86-a4af-e3b52d40bfd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52173f24f42a481380b047abe76bca57'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:17.607048 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ebce591e-ae55-4b86-a4af-e3b52d40bfd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52173f24f42a481380b047abe76bca57'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:17.614047 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1472/2920] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:03:17 2022] DELETE /identity/v3/users/52173f24f42a481380b047abe76bca57 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 19 15:03:17.796489 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130c7e1-6c12-47b7-af6e-0da0cf107d32 service placement] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:03:17.812729 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f130c7e1-6c12-47b7-af6e-0da0cf107d32 service placement] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:03:17.869367 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:03:17.870000 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:03:17.870275 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:03:17.870544 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a admin admin] PATH_INFO: `/v3/projects/cbb13ea6de2a48e99ce93be06fd87830` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:03:17.872290 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbb13ea6de2a48e99ce93be06fd87830)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:03:17.874199 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:03:17.920293 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb13ea6de2a48e99ce93be06fd87830', 'request_id': 'req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:17.926368 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb13ea6de2a48e99ce93be06fd87830', 'request_id': 'req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:17.934579 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb13ea6de2a48e99ce93be06fd87830', 'request_id': 'req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:17.945059 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a admin admin] The token cache is being invalidate because project cbb13ea6de2a48e99ce93be06fd87830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:03:17.945371 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:17.945983 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:17.947509 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1449/2921] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:03:17 2022] DELETE /identity/v3/projects/cbb13ea6de2a48e99ce93be06fd87830 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:03:17.951649 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce591e-ae55-4b86-a4af-e3b52d40bfd5 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:03:17.953134 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ebce591e-ae55-4b86-a4af-e3b52d40bfd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:03:17.966314 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce591e-ae55-4b86-a4af-e3b52d40bfd5 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:03:18.011740 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:03:18.024753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc496dce-3ad4-42b7-a5fe-5439c22d3572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:03:18.025377 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc496dce-3ad4-42b7-a5fe-5439c22d3572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:03:18.025653 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc496dce-3ad4-42b7-a5fe-5439c22d3572 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:03:18.025930 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc496dce-3ad4-42b7-a5fe-5439c22d3572 admin admin] PATH_INFO: `/v3/users/ba481d9001c54a11946049a06185790a` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:03:18.027072 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc496dce-3ad4-42b7-a5fe-5439c22d3572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba481d9001c54a11946049a06185790a)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:03:18.029001 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc496dce-3ad4-42b7-a5fe-5439c22d3572 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:03:18.074925 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cc496dce-3ad4-42b7-a5fe-5439c22d3572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba481d9001c54a11946049a06185790a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:18.082007 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cc496dce-3ad4-42b7-a5fe-5439c22d3572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba481d9001c54a11946049a06185790a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:18.095368 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cc496dce-3ad4-42b7-a5fe-5439c22d3572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba481d9001c54a11946049a06185790a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:18.103251 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cc496dce-3ad4-42b7-a5fe-5439c22d3572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba481d9001c54a11946049a06185790a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:18.104092 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-cc496dce-3ad4-42b7-a5fe-5439c22d3572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba481d9001c54a11946049a06185790a'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:18.110774 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1473/2922] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:03:17 2022] DELETE /identity/v3/users/ba481d9001c54a11946049a06185790a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:03:18.281620 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:03:18.283049 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:03:18.296828 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9cbca-c6d4-4e0d-9e48-c48457c0cf2a admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:03:18.305497 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:03:18.359755 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef5793-c7c4-4763-89ea-d7be74e4cf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:03:18.360414 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef5793-c7c4-4763-89ea-d7be74e4cf99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:03:18.360692 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef5793-c7c4-4763-89ea-d7be74e4cf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:03:18.360962 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef5793-c7c4-4763-89ea-d7be74e4cf99 admin admin] PATH_INFO: `/v3/projects/6b7766406094495b9d7eb12fcc35aee3` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:03:18.362075 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef5793-c7c4-4763-89ea-d7be74e4cf99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b7766406094495b9d7eb12fcc35aee3)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:03:18.364033 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef5793-c7c4-4763-89ea-d7be74e4cf99 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:03:18.410711 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-16ef5793-c7c4-4763-89ea-d7be74e4cf99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b7766406094495b9d7eb12fcc35aee3', 'request_id': 'req-16ef5793-c7c4-4763-89ea-d7be74e4cf99'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:18.417148 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-16ef5793-c7c4-4763-89ea-d7be74e4cf99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b7766406094495b9d7eb12fcc35aee3', 'request_id': 'req-16ef5793-c7c4-4763-89ea-d7be74e4cf99'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:18.426090 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-16ef5793-c7c4-4763-89ea-d7be74e4cf99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b7766406094495b9d7eb12fcc35aee3', 'request_id': 'req-16ef5793-c7c4-4763-89ea-d7be74e4cf99'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:18.437122 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-16ef5793-c7c4-4763-89ea-d7be74e4cf99 admin admin] The token cache is being invalidate because project 6b7766406094495b9d7eb12fcc35aee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:03:18.437408 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-16ef5793-c7c4-4763-89ea-d7be74e4cf99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:18.438033 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-16ef5793-c7c4-4763-89ea-d7be74e4cf99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:18.439490 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1450/2923] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:03:18 2022] DELETE /identity/v3/projects/6b7766406094495b9d7eb12fcc35aee3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:03:28.107728 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:03:28.434058 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:03:51.941074 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdd4f2-acb2-4418-851a-22650e2c0537 None None] REQUEST_METHOD: `POST` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:03:51.941591 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdd4f2-acb2-4418-851a-22650e2c0537 None None] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:03:51.941700 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdd4f2-acb2-4418-851a-22650e2c0537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:03:51.990745 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.auth.core [None req-8ccdd4f2-acb2-4418-851a-22650e2c0537 None None] MFA Rules not processed for user `0fefb7fa6f484341a19a1a8bee31b116`. Rule list: `[]` (Enabled: `True`). {{(pid=87873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 19 15:03:51.991989 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8ccdd4f2-acb2-4418-851a-22650e2c0537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:03:52.059614 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1474/2924] 10.222.0.60 () {58 vars in 967 bytes} [Mon Sep 19 15:03:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 19 15:03:52.103886 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef5793-c7c4-4763-89ea-d7be74e4cf99 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:03:52.105303 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-16ef5793-c7c4-4763-89ea-d7be74e4cf99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:03:52.198670 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef5793-c7c4-4763-89ea-d7be74e4cf99 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:03:52.203676 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b0783-4f0c-4402-b44b-56557453b8b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfc14d32f4314c6e93d20a9c5a45c220', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:03:52.204319 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b0783-4f0c-4402-b44b-56557453b8b9 service swift] REQUEST_METHOD: `GET` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:03:52.204600 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b0783-4f0c-4402-b44b-56557453b8b9 service swift] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:03:52.204869 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b0783-4f0c-4402-b44b-56557453b8b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:03:52.212827 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b0783-4f0c-4402-b44b-56557453b8b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:03:52.214409 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b0783-4f0c-4402-b44b-56557453b8b9 service swift] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:03:52.222434 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1451/2925] 10.222.0.60 () {60 vars in 1384 bytes} [Mon Sep 19 15:03:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 19 15:03:59.088979 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdd4f2-acb2-4418-851a-22650e2c0537 None None] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:03:59.091556 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-8ccdd4f2-acb2-4418-851a-22650e2c0537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:03:59.163847 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdd4f2-acb2-4418-851a-22650e2c0537 None None] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:03:59.168336 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11cd81-b465-45ec-9f63-4afb42a0c500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:03:59.168976 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11cd81-b465-45ec-9f63-4afb42a0c500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:03:59.169241 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11cd81-b465-45ec-9f63-4afb42a0c500 admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:03:59.169515 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11cd81-b465-45ec-9f63-4afb42a0c500 admin admin] PATH_INFO: `/v3/users/a3d7fd2980ce48a49931642b01bab212` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:03:59.170628 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11cd81-b465-45ec-9f63-4afb42a0c500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3d7fd2980ce48a49931642b01bab212)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:03:59.172612 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11cd81-b465-45ec-9f63-4afb42a0c500 admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:03:59.218625 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ee11cd81-b465-45ec-9f63-4afb42a0c500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3d7fd2980ce48a49931642b01bab212'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:59.225193 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ee11cd81-b465-45ec-9f63-4afb42a0c500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3d7fd2980ce48a49931642b01bab212'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:59.236955 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ee11cd81-b465-45ec-9f63-4afb42a0c500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3d7fd2980ce48a49931642b01bab212'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:59.238594 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:03:59.246080 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ee11cd81-b465-45ec-9f63-4afb42a0c500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3d7fd2980ce48a49931642b01bab212'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:59.247139 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-ee11cd81-b465-45ec-9f63-4afb42a0c500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3d7fd2980ce48a49931642b01bab212'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:59.254645 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1475/2926] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:03:59 2022] DELETE /identity/v3/users/a3d7fd2980ce48a49931642b01bab212 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:03:59.428888 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b0783-4f0c-4402-b44b-56557453b8b9 service swift] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:03:59.441357 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b0783-4f0c-4402-b44b-56557453b8b9 service swift] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:03:59.499687 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4efce8-c44a-4e41-81ae-a8c3b4446548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:03:59.500339 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4efce8-c44a-4e41-81ae-a8c3b4446548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:03:59.500622 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4efce8-c44a-4e41-81ae-a8c3b4446548 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:03:59.500897 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4efce8-c44a-4e41-81ae-a8c3b4446548 admin admin] PATH_INFO: `/v3/projects/5fca221e257f44dea6b74f7737978739` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:03:59.502015 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4efce8-c44a-4e41-81ae-a8c3b4446548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fca221e257f44dea6b74f7737978739)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:03:59.503953 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4efce8-c44a-4e41-81ae-a8c3b4446548 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:03:59.551341 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cf4efce8-c44a-4e41-81ae-a8c3b4446548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fca221e257f44dea6b74f7737978739', 'request_id': 'req-cf4efce8-c44a-4e41-81ae-a8c3b4446548'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:59.558238 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cf4efce8-c44a-4e41-81ae-a8c3b4446548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fca221e257f44dea6b74f7737978739', 'request_id': 'req-cf4efce8-c44a-4e41-81ae-a8c3b4446548'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:59.566520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cf4efce8-c44a-4e41-81ae-a8c3b4446548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fca221e257f44dea6b74f7737978739', 'request_id': 'req-cf4efce8-c44a-4e41-81ae-a8c3b4446548'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:59.573631 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:03:59.578874 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cf4efce8-c44a-4e41-81ae-a8c3b4446548 admin admin] The token cache is being invalidate because project 5fca221e257f44dea6b74f7737978739 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:03:59.579371 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cf4efce8-c44a-4e41-81ae-a8c3b4446548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:59.580524 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-cf4efce8-c44a-4e41-81ae-a8c3b4446548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:59.582512 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1452/2927] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:03:59 2022] DELETE /identity/v3/projects/5fca221e257f44dea6b74f7737978739 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:03:59.586481 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11cd81-b465-45ec-9f63-4afb42a0c500 admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:03:59.588014 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-ee11cd81-b465-45ec-9f63-4afb42a0c500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:03:59.601302 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11cd81-b465-45ec-9f63-4afb42a0c500 admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:03:59.657972 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b3a49-42a6-4e7f-bea6-38c69081094d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:03:59.658591 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b3a49-42a6-4e7f-bea6-38c69081094d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:03:59.658869 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b3a49-42a6-4e7f-bea6-38c69081094d admin admin] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:03:59.659142 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b3a49-42a6-4e7f-bea6-38c69081094d admin admin] PATH_INFO: `/v3/users/361efb27f3ae421fb721931ab0318d84` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:03:59.660290 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b3a49-42a6-4e7f-bea6-38c69081094d admin admin] RBAC: Authorizing `identity:delete_user(user_id=361efb27f3ae421fb721931ab0318d84)` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:03:59.662180 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b3a49-42a6-4e7f-bea6-38c69081094d admin admin] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:03:59.709753 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a54b3a49-42a6-4e7f-bea6-38c69081094d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '361efb27f3ae421fb721931ab0318d84'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:59.716819 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a54b3a49-42a6-4e7f-bea6-38c69081094d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '361efb27f3ae421fb721931ab0318d84'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:59.728682 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a54b3a49-42a6-4e7f-bea6-38c69081094d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '361efb27f3ae421fb721931ab0318d84'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:59.736830 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a54b3a49-42a6-4e7f-bea6-38c69081094d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '361efb27f3ae421fb721931ab0318d84'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:59.737521 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.notifications [None req-a54b3a49-42a6-4e7f-bea6-38c69081094d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '361efb27f3ae421fb721931ab0318d84'} {{(pid=87873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:03:59.743465 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1476/2928] 10.222.0.60 () {58 vars in 1224 bytes} [Mon Sep 19 15:03:59 2022] DELETE /identity/v3/users/361efb27f3ae421fb721931ab0318d84 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:03:59.917053 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4efce8-c44a-4e41-81ae-a8c3b4446548 admin admin] Authenticating user token {{(pid=87872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:03:59.918495 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.fernet_utils [None req-cf4efce8-c44a-4e41-81ae-a8c3b4446548 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:03:59.932573 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4efce8-c44a-4e41-81ae-a8c3b4446548 admin admin] Validating token access rules against request {{(pid=87872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:03:59.991182 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23186fb-b228-457e-8d7d-dab515015fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d770189a7bb4b81b8a42702f35fc84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b206fdda52da4f339c6c008d4c7973af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:03:59.991818 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23186fb-b228-457e-8d7d-dab515015fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:03:59.992127 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23186fb-b228-457e-8d7d-dab515015fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:03:59.992402 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23186fb-b228-457e-8d7d-dab515015fa7 admin admin] PATH_INFO: `/v3/projects/677fa01f1558491dac05c307944f365d` {{(pid=87872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:03:59.993520 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23186fb-b228-457e-8d7d-dab515015fa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=677fa01f1558491dac05c307944f365d)` {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:03:59.995441 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23186fb-b228-457e-8d7d-dab515015fa7 admin admin] RBAC: Authorization granted {{(pid=87872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:04:00.041613 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e23186fb-b228-457e-8d7d-dab515015fa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '677fa01f1558491dac05c307944f365d', 'request_id': 'req-e23186fb-b228-457e-8d7d-dab515015fa7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:04:00.048488 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e23186fb-b228-457e-8d7d-dab515015fa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '677fa01f1558491dac05c307944f365d', 'request_id': 'req-e23186fb-b228-457e-8d7d-dab515015fa7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:04:00.056188 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e23186fb-b228-457e-8d7d-dab515015fa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '677fa01f1558491dac05c307944f365d', 'request_id': 'req-e23186fb-b228-457e-8d7d-dab515015fa7'} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:04:00.066974 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e23186fb-b228-457e-8d7d-dab515015fa7 admin admin] The token cache is being invalidate because project 677fa01f1558491dac05c307944f365d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 19 15:04:00.067264 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e23186fb-b228-457e-8d7d-dab515015fa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:04:00.067930 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG keystone.notifications [None req-e23186fb-b228-457e-8d7d-dab515015fa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 19 15:04:00.069410 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: [pid: 87872|app: 0|req: 1453/2929] 10.222.0.60 () {58 vars in 1233 bytes} [Mon Sep 19 15:03:59 2022] DELETE /identity/v3/projects/677fa01f1558491dac05c307944f365d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 19 15:04:09.739954 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=55 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:04:10.064055 dv-cinder-756709-7-iscsi devstack@keystone.service[87872]: DEBUG dbcounter [-] [87872] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 15:05:38.528346 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b3a49-42a6-4e7f-bea6-38c69081094d admin admin] Authenticating user token {{(pid=87873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 19 15:05:38.529856 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-a54b3a49-42a6-4e7f-bea6-38c69081094d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:05:38.600428 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b3a49-42a6-4e7f-bea6-38c69081094d admin admin] Validating token access rules against request {{(pid=87873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 19 15:05:38.663550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61f288-ce3d-43c3-b46c-6db0df961d7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '568189358b154b66a729fae2f0411d1b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7dee98edf19d478ca4f746ff85579fba', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 19 15:05:38.664187 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61f288-ce3d-43c3-b46c-6db0df961d7d service placement] REQUEST_METHOD: `GET` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 19 15:05:38.664473 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61f288-ce3d-43c3-b46c-6db0df961d7d service placement] SCRIPT_NAME: `/identity` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 19 15:05:38.664750 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61f288-ce3d-43c3-b46c-6db0df961d7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 19 15:05:38.666525 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.fernet_utils [None req-3a61f288-ce3d-43c3-b46c-6db0df961d7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 19 15:05:38.679471 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61f288-ce3d-43c3-b46c-6db0df961d7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 19 15:05:38.681403 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61f288-ce3d-43c3-b46c-6db0df961d7d service placement] RBAC: Authorization granted {{(pid=87873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 19 15:05:38.693099 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: [pid: 87873|app: 0|req: 1477/2930] 10.222.0.60 () {60 vars in 1365 bytes} [Mon Sep 19 15:05:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 19 15:05:48.688550 dv-cinder-756709-7-iscsi devstack@keystone.service[87873]: DEBUG dbcounter [-] [87873] Writing DB stats keystone:SELECT=58 {{(pid=87873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}